class: title, self-paced Introduction<br/>to Docker and<br/>Containers<br/> .nav[*Self-paced version*] .debug[ ``` ``` These slides have been built from commit: 7bd50cd [common/title.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/title.md)] --- class: title, in-person Introduction<br/>to Docker and<br/>Containers<br/><br/></br> .footnote[ **Be kind to the WiFi!**<br/> <!-- *Use the 5G network.* --> *Don't use your hotspot.*<br/> *Don't stream videos or download big files during the workshop.*<br/> *Thank you!* **Slides: http://container.training/** ] .debug[[common/title.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/title.md)] --- ## Intros - Hello! We are: - .emoji[👷🏻‍♀️] AJ ([@s0ulshake](https://twitter.com/s0ulshake), Travis CI) - .emoji[🐳] Jérôme ([@jpetazzo](https://twitter.com/jpetazzo), Docker Inc.) - The workshop will run from 9am to 4pm - There will be a lunch break at noon (And coffee breaks!) - Feel free to interrupt for questions at any time - *Especially when you see full screen container pictures!* - Live feedback, questions, help on [Gitter](https://gitter.im/jpetazzo/workshop-20171116-sanfrancisco) .debug[[logistics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/logistics.md)] --- ## A brief introduction - This was initially written to support in-person, instructor-led workshops and tutorials - You can also follow along on your own, at your own pace - We included as much information as possible in these slides - We recommend having a mentor to help you ... - ... Or be comfortable spending some time reading the Docker [documentation](https://docs.docker.com/) ... - ... And looking for answers in the [Docker forums](forums.docker.com), [StackOverflow](http://stackoverflow.com/questions/tagged/docker), and other outlets .debug[[common/intro.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/intro.md)] --- class: self-paced ## Hands on, you shall practice - Nobody ever became a Jedi by spending their lives reading Wookiepedia - Likewise, it will take more than merely *reading* these slides to make you an expert - These slides include *tons* of exercises and examples - They assume that you have access to some Docker nodes - If you are attending a workshop or tutorial: <br/>you will be given specific instructions to access your cluster - If you are doing this on your own: <br/>the first chapter will give you various options like [Play-With-Docker](http://www.play-with-docker.com/) to get your own cluster .debug[[common/intro.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/intro.md)] --- ## About these slides - All the content is available in a public GitHub repository: https://github.com/jpetazzo/container.training - You can get updated "builds" of the slides there: http://container.training/ - Typos? Mistakes? Questions? Feel free to hover over the bottom of the slide ... -- .footnote[.emoji[👇] Try it! The source file will be shown and you can view it on GitHub and fork and edit it.] .debug[[common/intro.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/intro.md)] --- name: toc-chapter-1 ## Chapter 1 - [Docker 30,000ft overview](#toc-docker-ft-overview) - [Our training environment](#toc-our-training-environment) - [Installing Docker](#toc-installing-docker) - [Our first containers](#toc-our-first-containers) - [Background containers](#toc-background-containers) - [Restarting and attaching to containers](#toc-restarting-and-attaching-to-containers) .debug[(auto-generated TOC)] --- name: toc-chapter-2 ## Chapter 2 - [Understanding Docker images](#toc-understanding-docker-images) - [Building images interactively](#toc-building-images-interactively) - [Building Docker images with a Dockerfile](#toc-building-docker-images-with-a-dockerfile) - [`CMD` and `ENTRYPOINT`](#toc-cmd-and-entrypoint) - [Copying files during the build](#toc-copying-files-during-the-build) - [Multi-stage builds](#toc-multi-stage-builds) - [Publishing images to the Docker Hub](#toc-publishing-images-to-the-docker-hub) - [Tips for efficient Dockerfiles](#toc-tips-for-efficient-dockerfiles) .debug[(auto-generated TOC)] --- name: toc-chapter-3 ## Chapter 3 - [Naming and inspecting containers](#toc-naming-and-inspecting-containers) - [Container networking basics](#toc-container-networking-basics) - [Container network drivers](#toc-container-network-drivers) - [The Container Network Model](#toc-the-container-network-model) - [Service discovery with containers](#toc-service-discovery-with-containers) - [Ambassadors](#toc-ambassadors) .debug[(auto-generated TOC)] --- name: toc-chapter-4 ## Chapter 4 - [Local development workflow with Docker](#toc-local-development-workflow-with-docker) - [Working with volumes](#toc-working-with-volumes) - [Compose for development stacks](#toc-compose-for-development-stacks) - [Advanced Dockerfiles](#toc-advanced-dockerfiles) - [Links and resources](#toc-links-and-resources) .debug[(auto-generated TOC)] .debug[[common/toc.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/toc.md)] --- class: pic .interstitial[] --- name: toc-docker-ft-overview class: title Docker 30,000ft overview .nav[ [Previous section](#toc-) | [Back to table of contents](#toc-chapter-1) | [Next section](#toc-our-training-environment) ] .debug[(automatically generated title slide)] --- # Docker 30,000ft overview In this lesson, we will learn about: * Why containers (non-technical elevator pitch) * Why containers (technical elevator pitch) * How Docker helps us to build, ship, and run * The history of containers We won't actually run Docker or containers in this chapter (yet!). Don't worry, we will get to that fast enough! .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- ## Elevator pitch ### (for your manager, your boss...) .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- ## OK... Why the buzz around containers? * The software industry has changed * Before: * monolithic applications * long development cycles * single environment * slowly scaling up * Now: * decoupled services * fast, iterative improvements * multiple environments * quickly scaling out .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- ## Deployment becomes very complex * Many different stacks: * languages * frameworks * databases * Many different targets: * individual development environments * pre-production, QA, staging... * production: on prem, cloud, hybrid .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: pic ## The deployment problem  .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: pic ## The matrix from hell  .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: pic ## The parallel with the shipping indsutry  .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: pic ## Intermodal shipping containers  .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: pic ## A new shipping ecosystem  .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: pic ## A shipping container system for applications  .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: pic ## Eliminate the matrix from hell  .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- ## Results * [Dev-to-prod reduced from 9 months to 15 minutes (ING)]( https://www.docker.com/sites/default/files/CS_ING_01.25.2015_1.pdf) * [Continuous integration job time reduced by more than 60% (BBC)]( https://www.docker.com/sites/default/files/CS_BBCNews_01.25.2015_1.pdf) * [Deploy 100 times a day instead of once a week (GILT)]( https://www.docker.com/sites/default/files/CS_Gilt%20Groupe_03.18.2015_0.pdf) * [70% infrastructure consolidation (MetLife)]( https://www.docker.com/customers/metlife-transforms-customer-experience-legacy-and-microservices-mashup) * [60% infrastructure consolidation (Intesa Sanpaolo)]( https://blog.docker.com/2017/11/intesa-sanpaolo-builds-resilient-foundation-banking-docker-enterprise-edition/) * [14x application density; 60% of legacy datacenter migrated in 4 months (GE Appliances)]( https://www.docker.com/customers/ge-uses-docker-enable-self-service-their-developers) * etc. .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- ## Elevator pitch ### (for your fellow devs and ops) .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- ## Escape dependency hell 1. Write installation instructions into an `INSTALL.txt` file 2. Using this file, write an `install.sh` script that works *for you* 3. Turn this file into a `Dockerfile`, test it on your machine 4. If the Dockerfile builds on your machine, it will build *anywhere* 5. Rejoice as you escape dependency hell and "works on my machine" Never again "worked in dev - ops problem now!" .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- ## On-board developers and contributors rapidly 1. Write Dockerfiles for your application components 2. Use pre-made images from the Docker Hub (mysql, redis...) 3. Describe your stack with a Compose file 4. On-board somebody with two commands: ```bash git clone ... docker-compose up ``` With this, you can create development, integration, QA environments in minutes! .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Implement reliable CI easily 1. Build test environment with a Dockerfile or Compose file 2. For each test run, stage up a new container or stack 3. Each run is now in a clean environment 4. No pollution from previous tests Way faster and cheaper than creating VMs each time! .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Use container images as build artefacts 1. Build your app from Dockerfiles 2. Store the resulting images in a registry 3. Keep them forever (or as long as necessary) 4. Test those images in QA, CI, integration... 5. Run the same images in production 6. Something goes wrong? Rollback to previous image 7. Investigating old regression? Old image has your back! Images contain all the libraries, dependencies, etc. needed to run the app. .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Decouple "plumbing" from application logic 1. Write your code to connect to named services ("db", "api"...) 2. Use Compose to start your stack 3. Docker will setup per-container DNS resolver for those names 4. You can now scale, add load balancers, replication ... without changing your code Note: this is not covered in this intro level workshop! .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## What did Docker bring to the table? ### Docker before/after .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Formats and APIs, before Docker * No standardized exchange format. <br/>(No, a rootfs tarball is *not* a format!) * Containers are hard to use for developers. <br/>(Where's the equivalent of `docker run debian`?) * As a result, they are *hidden* from the end users. * No re-usable components, APIs, tools. <br/>(At best: VM abstractions, e.g. libvirt.) Analogy: * Shipping containers are not just steel boxes. * They are steel boxes that are a standard size, with the same hooks and holes. .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Formats and APIs, after Docker * Standardize the container format, because containers were not portable. * Make containers easy to use for developers. * Emphasis on re-usable components, APIs, ecosystem of standard tools. * Improvement over ad-hoc, in-house, specific tools. .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Shipping, before Docker * Ship packages: deb, rpm, gem, jar, homebrew... * Dependency hell. * "Works on my machine." * Base deployment often done from scratch (debootstrap...) and unreliable. .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Shipping, after Docker * Ship container images with all their dependencies. * Images are bigger, but they are broken down into layers. * Only ship layers that have changed. * Save disk, network, memory usage. .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Example Layers: * CentOS * JRE * Tomcat * Dependencies * Application JAR * Configuration .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Devs vs Ops, before Docker * Drop a tarball (or a commit hash) with instructions. * Dev environment very different from production. * Ops don't always have a dev environment themselves ... * ... and when they do, it can differ from the devs'. * Ops have to sort out differences and make it work ... * ... or bounce it back to devs. * Shipping code causes frictions and delays. .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: extra-details ## Devs vs Ops, after Docker * Drop a container image or a Compose file. * Ops can always run that container image. * Ops can always run that Compose file. * Ops still have to adapt to prod environment, but at least they have a reference point. * Ops have tools allowing to use the same image in dev and prod. * Devs can be empowered to make releases themselves more easily. .debug[[intro/Docker_Overview.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Docker_Overview.md)] --- class: pic .interstitial[] --- name: toc-our-training-environment class: title Our training environment .nav[ [Previous section](#toc-docker-ft-overview) | [Back to table of contents](#toc-chapter-1) | [Next section](#toc-installing-docker) ] .debug[(automatically generated title slide)] --- class: title # Our training environment  .debug[[intro/Training_Environment.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Training_Environment.md)] --- ## Our training environment - If you are attending a tutorial or workshop: - a VM has been provisioned for each student - If you are doing or re-doing this course on your own, you can: - install Docker locally (as explained in the chapter "Installing Docker") - install Docker on e.g. a cloud VM - use http://www.play-with-docker.com/ to instantly get a training environment .debug[[intro/Training_Environment.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Training_Environment.md)] --- ## Our Docker VM *This section assumes that you are following this course as part of a tutorial, training or workshop, where each student is given an individual Docker VM.* - The VM is created just before the training. - It will stay up during the whole training. - It will be destroyed shortly after the training. - It comes pre-loaded with Docker and some other useful tools. .debug[[intro/Training_Environment.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Training_Environment.md)] --- ## Connecting to your Virtual Machine You need an SSH client. * On OS X, Linux, and other UNIX systems, just use `ssh`: ```bash $ ssh <login>@<ip-address> ``` * On Windows, if you don't have an SSH client, you can download: * Putty (www.putty.org) * Git BASH (https://git-for-windows.github.io/) * MobaXterm (http://moabaxterm.mobatek.net) .debug[[intro/Training_Environment.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Training_Environment.md)] --- ## Checking your Virtual Machine Once logged in, make sure that you can run a basic Docker command: .small[ ```bash $ docker version Client: Version: 17.09.0-ce API version: 1.32 Go version: go1.8.3 Git commit: afdb6d4 Built: Tue Sep 26 22:40:09 2017 OS/Arch: darwin/amd64 Server: Version: 17.09.0-ce API version: 1.32 (minimum version 1.12) Go version: go1.8.3 Git commit: afdb6d4 Built: Tue Sep 26 22:45:38 2017 OS/Arch: linux/amd64 Experimental: true ``` ] If this doesn't work, raise your hand so that an instructor can assist you! .debug[[intro/Training_Environment.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Training_Environment.md)] --- class: pic .interstitial[] --- name: toc-installing-docker class: title Installing Docker .nav[ [Previous section](#toc-our-training-environment) | [Back to table of contents](#toc-chapter-1) | [Next section](#toc-our-first-containers) ] .debug[(automatically generated title slide)] --- class: title # Installing Docker  .debug[[intro/Installing_Docker.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Installing_Docker.md)] --- ## Objectives At the end of this lesson, you will know: * How to install Docker. * When to use `sudo` when running Docker commands. *Note:* if you were provided with a training VM for a hands-on tutorial, you can skip this chapter, since that VM already has Docker installed, and Docker has already been setup to run without `sudo`. .debug[[intro/Installing_Docker.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Installing_Docker.md)] --- ## Installing Docker There are many ways to install Docker. We can arbitrarily distinguish: * Installing Docker on an existing Linux machine (physical or VM) * Installing Docker on MacOS or Windows * Installing Docker on a fleet of cloud VMs .debug[[intro/Installing_Docker.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Installing_Docker.md)] --- ## Installing Docker on Linux * The recommended method is to install the packages supplied by Docker Inc. * The general method is: - add Docker Inc.'s package repositories to your system configuration - install the Docker Engine * Detailed installation instructions (distro by distro) are available on: https://docs.docker.com/engine/installation/ * You can also install from binaries (if your distro is not supported): https://docs.docker.com/engine/installation/linux/docker-ce/binaries/ .debug[[intro/Installing_Docker.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Installing_Docker.md)] --- ## Installing Docker on MacOS and Windows * On MacOS, the recommended method is to use Docker4Mac: https://docs.docker.com/docker-for-mac/install/ * On Windows 10 Pro, Enterprise, and Eduction, you can use Docker4Windows: https://docs.docker.com/docker-for-windows/install/ * On older versions of Windows, you can use the Docker Toolbox: https://docs.docker.com/toolbox/toolbox_install_windows/ .debug[[intro/Installing_Docker.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Installing_Docker.md)] --- ## Running Docker on MacOS and Windows When you execute `docker version` from the terminal: * the CLI connects to the Docker Engine over a standard socket, * the Docker Engine is, in fact, running in a VM, * ... but the CLI doesn't know or care about that, * the CLI sends a request using the REST API, * the Docker Engine in the VM processes the request, * the CLI gets the response and displays it to you. All communication with the Docker Engine happens over the API. This will also allow to use remote Engines exactly as if they were local. .debug[[intro/Installing_Docker.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Installing_Docker.md)] --- ## Docker4Mac and Docker4Windows * They let you run Docker without VirtualBox * They are installed like normal applications (think QEMU, but faster) * They access network resources like normal applications <br/>(and therefore, play well with enterprise VPNs and firewalls) * They support filesystem sharing through volumes (we'll talk about this later) * They only support running one Docker VM at a time ... ... so if you want to run a full cluster locally, install e.g. the Docker Toolbox * They can co-exist with the Docker Toolbox .debug[[intro/Installing_Docker.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Installing_Docker.md)] --- ## Important PSA about security * If you have access to the Docker control socket, you can take over the machine (Because you can run containers that will access the machine's resources) * Therefore, on Linux machines, the `docker` user is equivalent to `root` * You should restrict access to it like you would protect `root` * By default, the Docker control socket belongs to the `docker` group * You can add trusted users to the `docker` group * Otherwise, you will have to prefix every `docker` command with `sudo`, e.g.: ```bash sudo docker version ``` .debug[[intro/Installing_Docker.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Installing_Docker.md)] --- class: pic .interstitial[] --- name: toc-our-first-containers class: title Our first containers .nav[ [Previous section](#toc-installing-docker) | [Back to table of contents](#toc-chapter-1) | [Next section](#toc-background-containers) ] .debug[(automatically generated title slide)] --- class: title # Our first containers  .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Objectives At the end of this lesson, you will have: * Seen Docker in action. * Started your first containers. .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Hello World In your Docker environment, just run the following command: ```bash $ docker run busybox echo hello world hello world ``` (If your Docker install is brand new, you will also see a few extra lines, corresponding to the download of the `busybox` image.) .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## That was our first container! * We used one of the smallest, simplest images available: `busybox`. * `busybox` is typically used in embedded systems (phones, routers...) * We ran a single process and echo'ed `hello world`. .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## A more useful container Let's run a more exciting container: ```bash $ docker run -it ubuntu root@04c0bb0a6c07:/# ``` * This is a brand new container. * It runs a bare-bones, no-frills `ubuntu` system. * `-it` is shorthand for `-i -t`. * `-i` tells Docker to connect us to the container's stdin. * `-t` tells Docker that we want a pseudo-terminal. .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Do something in our container Try to run `figlet` in our container. ```bash root@04c0bb0a6c07:/# figlet hello bash: figlet: command not found ``` Alright, we need to install it. .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Install a package in our container We want `figlet`, so let's install it: ```bash root@04c0bb0a6c07:/# apt-get update ... Fetched 1514 kB in 14s (103 kB/s) Reading package lists... Done root@04c0bb0a6c07:/# apt-get install figlet Reading package lists... Done ... ``` One minute later, `figlet` is installed! .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Try to run our freshly installed program The `figlet` program takes a message as parameter. ```bash root@04c0bb0a6c07:/# figlet hello _ _ _ | |__ ___| | | ___ | '_ \ / _ \ | |/ _ \ | | | | __/ | | (_) | |_| |_|\___|_|_|\___/ ``` Beautiful! .emoji[😍] .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Counting packages in the container Let's check how many packages are installed there. ```bash root@04c0bb0a6c07:/# dpkg -l | wc -l 190 ``` * `dpkg -l` lists the packages installed in our container * `wc -l` counts them How many packages do we have on our host? .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Counting packages on the host Exit the container by logging out of the shell, like you would usually do. (E.g. with `^D` or `exit`) ```bash root@04c0bb0a6c07:/# exit ``` Now, try to: * run `dpkg -l | wc -l`. How many packages are installed? * run `figlet`. Does that work? .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Host and containers are independent things * We ran an `ubuntu` container on an `ubuntu` host. * But they have different, independent packages. * Installing something on the host doesn't expose it to the container. * And vice-versa. * We can run *any container* on *any host*. .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Where's our container? * Our container is now in a *stopped* state. * It still exists on disk, but all compute resources have been freed up. * We will see later how to get back to that container. .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- ## Starting another container What if we start a new container, and try to run `figlet` again? ```bash $ docker run -it ubuntu root@b13c164401fb:/# figlet bash: figlet: command not found ``` * We started a *brand new container*. * The basic Ubuntu image was used, and `figlet` is not here. * We will see in the next chapters how to bake a custom image with `figlet`. .debug[[intro/First_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/First_Containers.md)] --- class: pic .interstitial[] --- name: toc-background-containers class: title Background containers .nav[ [Previous section](#toc-our-first-containers) | [Back to table of contents](#toc-chapter-1) | [Next section](#toc-restarting-and-attaching-to-containers) ] .debug[(automatically generated title slide)] --- class: title # Background containers  .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Objectives Our first containers were *interactive*. We will now see how to: * Run a non-interactive container. * Run a container in the background. * List running containers. * Check the logs of a container. * Stop a container. * List stopped containers. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## A non-interactive container We will run a small custom container. This container just displays the time every second. ```bash $ docker run jpetazzo/clock Fri Feb 20 00:28:53 UTC 2015 Fri Feb 20 00:28:54 UTC 2015 Fri Feb 20 00:28:55 UTC 2015 ... ``` * This container will run forever. * To stop it, press `^C`. * Docker has automatically downloaded the image `jpetazzo/clock`. * This image is a user image, created by `jpetazzo`. * We will hear more about user images (and other types of images) later. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Run a container in the background Containers can be started in the background, with the `-d` flag (daemon mode): ```bash $ docker run -d jpetazzo/clock 47d677dcfba4277c6cc68fcaa51f932b544cab1a187c853b7d0caf4e8debe5ad ``` * We don't see the output of the container. * But don't worry: Docker collects that output and logs it! * Docker gives us the ID of the container. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## List running containers How can we check that our container is still running? With `docker ps`, just like the UNIX `ps` command, lists running processes. ```bash $ docker ps CONTAINER ID IMAGE ... CREATED STATUS ... 47d677dcfba4 jpetazzo/clock ... 2 minutes ago Up 2 minutes ... ``` Docker tells us: * The (truncated) ID of our container. * The image used to start the container. * That our container has been running (`Up`) for a couple of minutes. * Other information (COMMAND, PORTS, NAMES) that we will explain later. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Starting more containers Let's start two more containers. ```bash $ docker run -d jpetazzo/clock 57ad9bdfc06bb4407c47220cf59ce21585dce9a1298d7a67488359aeaea8ae2a ``` ```bash $ docker run -d jpetazzo/clock 068cc994ffd0190bbe025ba74e4c0771a5d8f14734af772ddee8dc1aaf20567d ``` Check that `docker ps` correctly reports all 3 containers. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Viewing only the last container started When many containers are already running, it can be useful to see only the last container that was started. This can be achieved with the `-l` ("Last") flag: ```bash $ docker ps -l CONTAINER ID IMAGE ... CREATED STATUS ... 068cc994ffd0 jpetazzo/clock ... 2 minutes ago Up 2 minutes ... ``` .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## View only the IDs of the containers Many Docker commands will work on container IDs: `docker stop`, `docker rm`... If we want to list only the IDs of our containers (without the other colums or the header line), we can use the `-q` ("Quiet", "Quick") flag: ```bash $ docker ps -q 068cc994ffd0 57ad9bdfc06b 47d677dcfba4 ``` .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Combining flags We can combine `-l` and `-q` to see only the ID of the last container started: ```bash $ docker ps -lq 068cc994ffd0 ``` At a first glance, it looks like this would be particularly useful in scripts. However, if we want to start a container and get its ID in a reliable way, it is better to use `docker run -d`, which we will cover in a bit. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## View the logs of a container We told you that Docker was logging the container output. Let's see that now. ```bash $ docker logs 068 Fri Feb 20 00:39:52 UTC 2015 Fri Feb 20 00:39:53 UTC 2015 ... ``` * We specified a *prefix* of the full container ID. * You can, of course, specify the full ID. * The `logs` command will output the *entire* logs of the container. <br/>(Sometimes, that will be too much. Let's see how to address that.) .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## View only the tail of the logs To avoid being spammed with eleventy pages of output, we can use the `--tail` option: ```bash $ docker logs --tail 3 068 Fri Feb 20 00:55:35 UTC 2015 Fri Feb 20 00:55:36 UTC 2015 Fri Feb 20 00:55:37 UTC 2015 ``` * The parameter is the number of lines that we want to see. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Follow the logs in real time Just like with the standard UNIX command `tail -f`, we can follow the logs of our container: ```bash $ docker logs --tail 1 --follow 068 Fri Feb 20 00:57:12 UTC 2015 Fri Feb 20 00:57:13 UTC 2015 ^C ``` * This will display the last line in the log file. * Then, it will continue to display the logs in real time. * Use `^C` to exit. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Stop our container There are two ways we can terminate our detached container. * Killing it using the `docker kill` command. * Stopping it using the `docker stop` command. The first one stops the container immediately, by using the `KILL` signal. The second one is more graceful. It sends a `TERM` signal, and after 10 seconds, if the container has not stopped, it sends `KILL.` Reminder: the `KILL` signal cannot be intercepted, and will forcibly terminate the container. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Stopping our containers Let's stop one of those containers: ```bash $ docker stop 47d6 47d6 ``` This will take 10 seconds: * Docker sends the TERM signal; * the container doesn't react to this signal (it's a simple Shell script with no special signal handling); * 10 seconds later, since the container is still running, Docker sends the KILL signal; * this terminates the container. .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## Killing the remaining containers Let's be less patient with the two other containers: ```bash $ docker kill 068 57ad 068 57ad ``` The `stop` and `kill` commands can take multiple container IDs. Those containers will be terminated immediately (without the 10 seconds delay). Let's check that our containers don't show up anymore: ```bash $ docker ps ``` .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- ## List stopped containers We can also see stopped containers, with the `-a` (`--all`) option. ```bash $ docker ps -a CONTAINER ID IMAGE ... CREATED STATUS 068cc994ffd0 jpetazzo/clock ... 21 min. ago Exited (137) 3 min. ago 57ad9bdfc06b jpetazzo/clock ... 21 min. ago Exited (137) 3 min. ago 47d677dcfba4 jpetazzo/clock ... 23 min. ago Exited (137) 3 min. ago 5c1dfd4d81f1 jpetazzo/clock ... 40 min. ago Exited (0) 40 min. ago b13c164401fb ubuntu ... 55 min. ago Exited (130) 53 min. ago ``` .debug[[intro/Background_Containers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Background_Containers.md)] --- class: pic .interstitial[] --- name: toc-restarting-and-attaching-to-containers class: title Restarting and attaching to containers .nav[ [Previous section](#toc-background-containers) | [Back to table of contents](#toc-chapter-1) | [Next section](#toc-understanding-docker-images) ] .debug[(automatically generated title slide)] --- # Restarting and attaching to containers We have started containers in the foreground, and in the background. In this chapter, we will see how to: * Put a container in the background. * Attach to a background container to bring it to the foreground. * Restart a stopped container. .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- ## Background and foreground The distinction between foreground and background containers is arbitrary. From Docker's point of view, all containers are the same. All containers run the same way, whether there is a client attached to them or not. It is always possible to detach from a container, and to reattach to a container. Analogy: attaching to a container is like plugging a keyboard and screen to a physical server. .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- ## Detaching from a container * If you have started an *interactive* container (with option `-it`), you can detach from it. * The "detach" sequence is `^P^Q`. * Otherwise you can detach by killing the Docker client. (But not by hitting `^C`, as this would deliver `SIGINT` to the container.) What does `-it` stand for? * `-t` means "allocate a terminal." * `-i` means "connect stdin to the terminal." .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- class: extra-details ## Specifying a custom detach sequence * You don't like `^P^Q`? No problem! * You can change the sequence with `docker run --detach-keys`. * This can also be passed as a global option to the engine. Start a container with a custom detach command: ```bash $ docker run -ti --detach-keys ctrl-x,x jpetazzo/clock ``` Detach by hitting `^X x`. (This is ctrl-x then x, not ctrl-x twice!) Check that our container is still running: ```bash $ docker ps -l ``` .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- class: extra-details ## Attaching to a container You can attach to a container: ```bash $ docker attach <containerID> ``` * The container must be running. * There *can* be multiple clients attached to the same container. * If you don't specify `--detach-keys` when attaching, it defaults back to `^P^Q`. Try it on our previous container: ```bash $ docker attach $(docker ps -lq) ``` Check that `^X x` doesn't work, but `^P ^Q` does. .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- ## Detaching from non-interactive containers * **Warning:** if the container was started without `-it`... * You won't be able to detach with `^P^Q`. * If you hit `^C`, the signal will be proxied to the container. * Remember: you can always detach by killing the Docker client. .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- ## Checking container output * Use `docker attach` if you intend to send input to the container. * If you just want to see the output of a container, use `docker logs`. ```bash $ docker logs --tail 1 --follow <containerID> ``` .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- ## Restarting a container When a container has exited, it is in stopped state. It can then be restarted with the `start` command. ```bash $ docker start <yourContainerID> ``` The container will be restarted using the same options you launched it with. You can re-attach to it if you want to interact with it: ```bash $ docker attach <yourContainerID> ``` Use `docker ps -a` to identify the container ID of a previous `jpetazzo/clock` container, and try those commands. .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- ## Attaching to a REPL * REPL = Read Eval Print Loop * Shells, interpreters, TUI ... * Symptom: you `docker attach`, and see nothing * The REPL doesn't know that you just attached, and doesn't print anything * Try hitting `^L` or `Enter` .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- class: extra-details ## SIGWINCH * When you `docker attach`, the Docker Engine sends SIGWINCH signals to the container. * SIGWINCH = WINdow CHange; indicates a change in window size. * This will cause some CLI and TUI programs to redraw the screen. * But not all of them. .debug[[intro/Start_And_Attach.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Start_And_Attach.md)] --- class: pic .interstitial[] --- name: toc-understanding-docker-images class: title Understanding Docker images .nav[ [Previous section](#toc-restarting-and-attaching-to-containers) | [Back to table of contents](#toc-chapter-2) | [Next section](#toc-building-images-interactively) ] .debug[(automatically generated title slide)] --- class: title # Understanding Docker images  .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Objectives In this section, we will explain: * What is an image. * What is a layer. * The various image namespaces. * How to search and download images. * Image tags and when to use them. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## What is an image? * Image = files + metadata * These files form the root filesystem of our container. * The metadata can indicate a number of things, e.g.: * the author of the image * the command to execute in the container when starting it * environment variables to be set * etc. * Images are made of *layers*, conceptually stacked on top of each other. * Each layer can add, change, and remove files and/or metadata. * Images can share layers to optimize disk usage, transfer times, and memory use. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Example for a Java webapp * CentOS base layer * Packages and configuration files added by our local IT * JRE * Tomcat * Our application's dependencies * Our application code and assets * Our application configuration .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Differences between containers and images * An image is a read-only filesystem. * A container is an encapsulated set of processes running in a read-write copy of that filesystem. * To optimize container boot time, *copy-on-write* is used instead of regular copy. * `docker run` starts a container from a given image. Let's give a couple of metaphors to illustrate those concepts. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Image as stencils Images are like templates or stencils that you can create containers from.  .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Object-oriented programming * Images are conceptually similar to *classes*. * Layers are conceptually similar to *inheritance*. * Containers are conceptually similar to *instances*. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Wait a minute... If an image is read-only, how do we change it? * We don't. * We create a new container from that image. * Then we make changes to that container. * When we are satisfied with those changes, we transform them into a new layer. * A new image is created by stacking the new layer on top of the old image. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## A chicken-and-egg problem * The only way to create an image is by "freezing" a container. * The only way to create a container is by instanciating an image. * Help! .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Creating the first images There is a special empty image called `scratch`. * It allows to *build from scratch*. The `docker import` command loads a tarball into Docker. * The imported tarball becomes a standalone image. * That new image has a single layer. Note: you will probably never have to do this yourself. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Creating other images `docker commit` * Saves all the changes made to a container into a new layer. * Creates a new image (effectively a copy of the container). `docker build` * Performs a repeatable build sequence. * This is the preferred method! We will explain both methods in a moment. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Images namespaces There are three namespaces: * Official images e.g. `ubuntu`, `busybox` ... * User (and organizations) images e.g. `jpetazzo/clock` * Self-hosted images e.g. `registry.example.com:5000/my-private/image` Let's explain each of them. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Root namespace The root namespace is for official images. They are put there by Docker Inc., but they are generally authored and maintained by third parties. Those images include: * Small, "swiss-army-knife" images like busybox. * Distro images to be used as bases for your builds, like ubuntu, fedora... * Ready-to-use components and services, like redis, postgresql... .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## User namespace The user namespace holds images for Docker Hub users and organizations. For example: ```bash jpetazzo/clock ``` The Docker Hub user is: ```bash jpetazzo ``` The image name is: ```bash clock ``` .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Self-Hosted namespace This namespace holds images which are not hosted on Docker Hub, but on third party registries. They contain the hostname (or IP address), and optionally the port, of the registry server. For example: ```bash localhost:5000/wordpress ``` * `localhost:5000` is the host and port of the registry * `wordpress` is the name of the image .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## How do you store and manage images? Images can be stored: * On your Docker host. * In a Docker registry. You can use the Docker client to download (pull) or upload (push) images. To be more accurate: you can use the Docker client to tell a Docker Engine to push and pull images to and from a registry. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Showing current images Let's look at what images are on our host now. ```bash $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE fedora latest ddd5c9c1d0f2 3 days ago 204.7 MB centos latest d0e7f81ca65c 3 days ago 196.6 MB ubuntu latest 07c86167cdc4 4 days ago 188 MB redis latest 4f5f397d4b7c 5 days ago 177.6 MB postgres latest afe2b5e1859b 5 days ago 264.5 MB alpine latest 70c557e50ed6 5 days ago 4.798 MB debian latest f50f9524513f 6 days ago 125.1 MB busybox latest 3240943c9ea3 2 weeks ago 1.114 MB training/namer latest 902673acc741 9 months ago 289.3 MB jpetazzo/clock latest 12068b93616f 12 months ago 2.433 MB ``` .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Searching for images We cannot list *all* images on a remote registry, but we can search for a specific keyword: ```bash $ docker search marathon NAME DESCRIPTION STARS OFFICIAL AUTOMATED mesosphere/marathon A cluster-wide init and co... 105 [OK] mesoscloud/marathon Marathon 31 [OK] mesosphere/marathon-lb Script to update haproxy b... 22 [OK] tobilg/mongodb-marathon A Docker image to start a ... 4 [OK] ``` * "Stars" indicate the popularity of the image. * "Official" images are those in the root namespace. * "Automated" images are built automatically by the Docker Hub. <br/>(This means that their build recipe is always available.) .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Downloading images There are two ways to download images. * Explicitly, with `docker pull`. * Implicitly, when executing `docker run` and the image is not found locally. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Pulling an image ```bash $ docker pull debian:jessie Pulling repository debian b164861940b8: Download complete b164861940b8: Pulling image (jessie) from debian d1881793a057: Download complete ``` * As seen previously, images are made up of layers. * Docker has downloaded all the necessary layers. * In this example, `:jessie` indicates which exact version of Debian we would like. It is a *version tag*. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Image and tags * Images can have tags. * Tags define image versions or variants. * `docker pull ubuntu` will refer to `ubuntu:latest`. * The `:latest` tag is generally updated often. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## When to (not) use tags Don't specify tags: * When doing rapid testing and prototyping. * When experimenting. * When you want the latest version. Do specify tags: * When recording a procedure into a script. * When going to production. * To ensure that the same version will be used everywhere. * To ensure repeatability later. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- ## Section summary We've learned how to: * Understand images and layers. * Understand Docker image namespacing. * Search and download images. .debug[[intro/Initial_Images.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Initial_Images.md)] --- class: pic .interstitial[] --- name: toc-building-images-interactively class: title Building images interactively .nav[ [Previous section](#toc-understanding-docker-images) | [Back to table of contents](#toc-chapter-2) | [Next section](#toc-building-docker-images-with-a-dockerfile) ] .debug[(automatically generated title slide)] --- # Building images interactively In this section, we will create our first container image. It will be a basic distribution image, but we will pre-install the package `figlet`. We will: * Create a container from a base image. * Install software manually in the container, and turn it into a new image. * Learn about new commands: `docker commit`, `docker tag`, and `docker diff`. .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- ## The plan 1. Create a container (with `docker run`) using our base distro of choice. 2. Run a bunch of commands to install and set up our software in the container. 3. (Optionally) review changes in the container with `docker diff`. 4. Turn the container into a new image with `docker commit`. 5. (Optionally) add tags to the image with `docker tag`. .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- ## Setting up our container Start an Ubuntu container: ```bash $ docker run -it ubuntu root@<yourContainerId>:#/ ``` Run the command `apt-get update` to refresh the list of packages available to install. Then run the command `apt-get install figlet` to install the program we are interested in. ```bash root@<yourContainerId>:#/ apt-get update && apt-get install figlet .... OUTPUT OF APT-GET COMMANDS .... ``` .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- ## Inspect the changes Type `exit` at the container prompt to leave the interactive session. Now let's run `docker diff` to see the difference between the base image and our container. ```bash $ docker diff <yourContainerId> C /root A /root/.bash_history C /tmp C /usr C /usr/bin A /usr/bin/figlet ... ``` .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- class: x-extra-details ## Docker tracks filesystem changes As explained before: * An image is read-only. * When we make changes, they happen in a copy of the image. * Docker can show the difference between the image, and its copy. * For performance, Docker uses copy-on-write systems. <br/>(i.e. starting a container based on a big image doesn't incur a huge copy.) .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- ## Copy-on-write security benefits * `docker diff` gives us an easy way to audit changes (à la Tripwire) * Containers can also be started in read-only mode (their root filesystem will be read-only, but they can still have read-write data volumes) .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- ## Commit our changes into a new image The `docker commit` command will create a new layer with those changes, and a new image using this new layer. ```bash $ docker commit <yourContainerId> <newImageId> ``` The output of the `docker commit` command will be the ID for your newly created image. We can use it as an argument to `docker run`. .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- ## Testing our new image Let's run this image: ```bash $ docker run -it <newImageId> root@fcfb62f0bfde:/# figlet hello _ _ _ | |__ ___| | | ___ | '_ \ / _ \ | |/ _ \ | | | | __/ | | (_) | |_| |_|\___|_|_|\___/ ``` It works! .emoji[🎉] .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- ## Tagging images Referring to an image by its ID is not convenient. Let's tag it instead. We can use the `tag` command: ```bash $ docker tag <newImageId> figlet ``` But we can also specify the tag as an extra argument to `commit`: ```bash $ docker commit <containerId> figlet ``` And then run it using its tag: ```bash $ docker run -it figlet ``` .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- ## What's next? Manual process = bad. Automated process = good. In the next chapter, we will learn how to automate the build process by writing a `Dockerfile`. .debug[[intro/Building_Images_Interactively.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_Interactively.md)] --- class: pic .interstitial[] --- name: toc-building-docker-images-with-a-dockerfile class: title Building Docker images with a Dockerfile .nav[ [Previous section](#toc-building-images-interactively) | [Back to table of contents](#toc-chapter-2) | [Next section](#toc-cmd-and-entrypoint) ] .debug[(automatically generated title slide)] --- class: title # Building Docker images with a Dockerfile  .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Objectives We will build a container image automatically, with a `Dockerfile`. At the end of this lesson, you will be able to: * Write a `Dockerfile`. * Build an image from a `Dockerfile`. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## `Dockerfile` overview * A `Dockerfile` is a build recipe for a Docker image. * It contains a series of instructions telling Docker how an image is constructed. * The `docker build` command builds an image from a `Dockerfile`. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Writing our first `Dockerfile` Our Dockerfile must be in a **new, empty directory**. 1. Create a directory to hold our `Dockerfile`. ```bash $ mkdir myimage ``` 2. Create a `Dockerfile` inside this directory. ```bash $ cd myimage $ vim Dockerfile ``` Of course, you can use any other editor of your choice. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Type this into our Dockerfile... ```dockerfile FROM ubuntu RUN apt-get update RUN apt-get install figlet ``` * `FROM` indicates the base image for our build. * Each `RUN` line will be executed by Docker during the build. * Our `RUN` commands **must be non-interactive.** <br/>(No input can be provided to Docker during the build.) * In many cases, we will add the `-y` flag to `apt-get`. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Build it! Save our file, then execute: ```bash $ docker build -t figlet . ``` * `-t` indicates the tag to apply to the image. * `.` indicates the location of the *build context*. We will talk more about the build context later. To keep things simple for now: this is the directory where our Dockerfile is located. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## What happens when we build the image? The output of `docker build` looks like this: .small[ ```bash $ docker build -t figlet . Sending build context to Docker daemon 2.048 kB Sending build context to Docker daemon Step 0 : FROM ubuntu ---> e54ca5efa2e9 Step 1 : RUN apt-get update ---> Running in 840cb3533193 ---> 7257c37726a1 Removing intermediate container 840cb3533193 Step 2 : RUN apt-get install figlet ---> Running in 2b44df762a2f ---> f9e8f1642759 Removing intermediate container 2b44df762a2f Successfully built f9e8f1642759 ``` ] * The output of the `RUN` commands has been omitted. * Let's explain what this output means. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Sending the build context to Docker ```bash Sending build context to Docker daemon 2.048 kB ``` * The build context is the `.` directory given to `docker build`. * It is sent (as an archive) by the Docker client to the Docker daemon. * This allows to use a remote machine to build using local files. * Be careful (or patient) if that directory is big and your link is slow. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Executing each step ```bash Step 1 : RUN apt-get update ---> Running in 840cb3533193 (...output of the RUN command...) ---> 7257c37726a1 Removing intermediate container 840cb3533193 ``` * A container (`840cb3533193`) is created from the base image. * The `RUN` command is executed in this container. * The container is committed into an image (`7257c37726a1`). * The build container (`840cb3533193`) is removed. * The output of this step will be the base image for the next one. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## The caching system If you run the same build again, it will be instantaneous. Why? * After each build step, Docker takes a snapshot of the resulting image. * Before executing a step, Docker checks if it has already built the same sequence. * Docker uses the exact strings defined in your Dockerfile, so: * `RUN apt-get install figlet cowsay ` <br/> is different from <br/> `RUN apt-get install cowsay figlet` * `RUN apt-get update` is not re-executed when the mirrors are updated You can force a rebuild with `docker build --no-cache ...`. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Running the image The resulting image is not different from the one produced manually. ```bash $ docker run -ti figlet root@91f3c974c9a1:/# figlet hello _ _ _ | |__ ___| | | ___ | '_ \ / _ \ | |/ _ \ | | | | __/ | | (_) | |_| |_|\___|_|_|\___/ ``` Yay! .emoji[🎉] .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Using image and viewing history The `history` command lists all the layers composing an image. For each layer, it shows its creation time, size, and creation command. When an image was built with a Dockerfile, each layer corresponds to a line of the Dockerfile. ```bash $ docker history figlet IMAGE CREATED CREATED BY SIZE f9e8f1642759 About an hour ago /bin/sh -c apt-get install fi 1.627 MB 7257c37726a1 About an hour ago /bin/sh -c apt-get update 21.58 MB 07c86167cdc4 4 days ago /bin/sh -c #(nop) CMD ["/bin 0 B <missing> 4 days ago /bin/sh -c sed -i 's/^#\s*\( 1.895 kB <missing> 4 days ago /bin/sh -c echo '#!/bin/sh' 194.5 kB <missing> 4 days ago /bin/sh -c #(nop) ADD file:b 187.8 MB ``` .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Introducing JSON syntax Most Dockerfile arguments can be passed in two forms: * plain string: <br/>`RUN apt-get install figlet` * JSON list: <br/>`RUN ["apt-get", "install", "figlet"]` We are going to change our Dockerfile to see how it affects the resulting image. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## Using JSON syntax in our Dockerfile Let's change our Dockerfile as follows! ```dockerfile FROM ubuntu RUN apt-get update RUN ["apt-get", "install", "figlet"] ``` Then build the new Dockerfile. ```bash $ docker build -t figlet . ``` .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## JSON syntax vs string syntax Compare the new history: ```bash $ docker history figlet IMAGE CREATED CREATED BY SIZE 27954bb5faaf 10 seconds ago apt-get install figlet 1.627 MB 7257c37726a1 About an hour ago /bin/sh -c apt-get update 21.58 MB 07c86167cdc4 4 days ago /bin/sh -c #(nop) CMD ["/bin 0 B <missing> 4 days ago /bin/sh -c sed -i 's/^#\s*\( 1.895 kB <missing> 4 days ago /bin/sh -c echo '#!/bin/sh' 194.5 kB <missing> 4 days ago /bin/sh -c #(nop) ADD file:b 187.8 MB ``` * JSON syntax specifies an *exact* command to execute. * String syntax specifies a command to be wrapped within `/bin/sh -c "..."`. .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- ## When to use JSON syntax and string syntax * String syntax: * is easier to write * interpolates environment variables and other shell expressions * creates an extra process (`/bin/sh -c ...`) to parse the string * requires `/bin/sh` to exist in the container * JSON syntax: * is harder to write (and read!) * passes all arguments without extra processing * doesn't create an extra process * doesn't require `/bin/sh` to exist in the container .debug[[intro/Building_Images_With_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Building_Images_With_Dockerfiles.md)] --- class: pic .interstitial[] --- name: toc-cmd-and-entrypoint class: title `CMD` and `ENTRYPOINT` .nav[ [Previous section](#toc-building-docker-images-with-a-dockerfile) | [Back to table of contents](#toc-chapter-2) | [Next section](#toc-copying-files-during-the-build) ] .debug[(automatically generated title slide)] --- class: title # `CMD` and `ENTRYPOINT`  .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Objectives In this lesson, we will learn about two important Dockerfile commands: `CMD` and `ENTRYPOINT`. These commands allow us to set the default command to run in a container. .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Defining a default command When people run our container, we want to greet them with a nice hello message, and using a custom font. For that, we will execute: ```bash figlet -f script hello ``` * `-f script` tells figlet to use a fancy font. * `hello` is the message that we want it to display. .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Adding `CMD` to our Dockerfile Our new Dockerfile will look like this: ```dockerfile FROM ubuntu RUN apt-get update RUN ["apt-get", "install", "figlet"] CMD figlet -f script hello ``` * `CMD` defines a default command to run when none is given. * It can appear at any point in the file. * Each `CMD` will replace and override the previous one. * As a result, while you can have multiple `CMD` lines, it is useless. .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Build and test our image Let's build it: ```bash $ docker build -t figlet . ... Successfully built 042dff3b4a8d ``` And run it: ```bash $ docker run figlet _ _ _ | | | | | | | | _ | | | | __ |/ \ |/ |/ |/ / \_ | |_/|__/|__/|__/\__/ ``` .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Overriding `CMD` If we want to get a shell into our container (instead of running `figlet`), we just have to specify a different program to run: ```bash $ docker run -it figlet bash root@7ac86a641116:/# ``` * We specified `bash`. * It replaced the value of `CMD`. .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Using `ENTRYPOINT` We want to be able to specify a different message on the command line, while retaining `figlet` and some default parameters. In other words, we would like to be able to do this: ```bash $ docker run figlet salut _ | | , __, | | _|_ / \_/ | |/ | | | \/ \_/|_/|__/ \_/|_/|_/ ``` We will use the `ENTRYPOINT` verb in Dockerfile. .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Adding `ENTRYPOINT` to our Dockerfile Our new Dockerfile will look like this: ```dockerfile FROM ubuntu RUN apt-get update RUN ["apt-get", "install", "figlet"] ENTRYPOINT ["figlet", "-f", "script"] ``` * `ENTRYPOINT` defines a base command (and its parameters) for the container. * The command line arguments are appended to those parameters. * Like `CMD`, `ENTRYPOINT` can appear anywhere, and replaces the previous value. Why did we use JSON syntax for our `ENTRYPOINT`? .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Implications of JSON vs string syntax * When CMD or ENTRYPOINT use string syntax, they get wrapped in `sh -c`. * To avoid this wrapping, we can use JSON syntax. What if we used `ENTRYPOINT` with string syntax? ```bash $ docker run figlet salut ``` This would run the following command in the `figlet` image: ```bash sh -c "figlet -f script" salut ``` .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Build and test our image Let's build it: ```bash $ docker build -t figlet . ... Successfully built 36f588918d73 ``` And run it: ```bash $ docker run figlet salut _ | | , __, | | _|_ / \_/ | |/ | | | \/ \_/|_/|__/ \_/|_/|_/ ``` .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Using `CMD` and `ENTRYPOINT` together What if we want to define a default message for our container? Then we will use `ENTRYPOINT` and `CMD` together. * `ENTRYPOINT` will define the base command for our container. * `CMD` will define the default parameter(s) for this command. * They *both* have to use JSON syntax. .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## `CMD` and `ENTRYPOINT` together Our new Dockerfile will look like this: ```dockerfile FROM ubuntu RUN apt-get update RUN ["apt-get", "install", "figlet"] ENTRYPOINT ["figlet", "-f", "script"] CMD ["hello world"] ``` * `ENTRYPOINT` defines a base command (and its parameters) for the container. * If we don't specify extra command-line arguments when starting the container, the value of `CMD` is appended. * Otherwise, our extra command-line arguments are used instead of `CMD`. .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Build and test our image Let's build it: ```bash $ docker build -t figlet . ... Successfully built 6e0b6a048a07 ``` Run it without parameters: ```bash $ docker run figlet _ _ _ _ | | | | | | | | | | | _ | | | | __ __ ,_ | | __| |/ \ |/ |/ |/ / \_ | | |_/ \_/ | |/ / | | |_/|__/|__/|__/\__/ \/ \/ \__/ |_/|__/\_/|_/ ``` .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Overriding the image default parameters Now let's pass extra arguments to the image. ```bash $ docker run figlet hola mundo _ _ | | | | | | | __ | | __, _ _ _ _ _ __| __ |/ \ / \_|/ / | / |/ |/ | | | / |/ | / | / \_ | |_/\__/ |__/\_/|_/ | | |_/ \_/|_/ | |_/\_/|_/\__/ ``` We overrode `CMD` but still used `ENTRYPOINT`. .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- ## Overriding `ENTRYPOINT` What if we want to run a shell in our container? We cannot just do `docker run figlet bash` because that would just tell figlet to display the word "bash." We use the `--entrypoint` parameter: ```bash $ docker run -it --entrypoint bash figlet root@6027e44e2955:/# ``` .debug[[intro/Cmd_And_Entrypoint.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Cmd_And_Entrypoint.md)] --- class: pic .interstitial[] --- name: toc-copying-files-during-the-build class: title Copying files during the build .nav[ [Previous section](#toc-cmd-and-entrypoint) | [Back to table of contents](#toc-chapter-2) | [Next section](#toc-multi-stage-builds) ] .debug[(automatically generated title slide)] --- class: title # Copying files during the build  .debug[[intro/Copying_Files_During_Build.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Copying_Files_During_Build.md)] --- ## Objectives So far, we have installed things in our container images by downloading packages. We can also copy files from the *build context* to the container that we are building. Remember: the *build context* is the directory containing the Dockerfile. In this chapter, we will learn a new Dockerfile keyword: `COPY`. .debug[[intro/Copying_Files_During_Build.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Copying_Files_During_Build.md)] --- ## Build some C code We want to build a container that compiles a basic "Hello world" program in C. Here is the program, `hello.c`: ```bash int main () { puts("Hello, world!"); return 0; } ``` Let's create a new directory, and put this file in there. Then we will write the Dockerfile. .debug[[intro/Copying_Files_During_Build.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Copying_Files_During_Build.md)] --- ## The Dockerfile On Debian and Ubuntu, the package `build-essential` will get us a compiler. When installing it, don't forget to specify the `-y` flag, otherwise the build will fail (since the build cannot be interactive). Then we will use `COPY` to place the source file into the container. ```bash FROM ubuntu RUN apt-get update RUN apt-get install -y build-essential COPY hello.c / RUN make hello CMD /hello ``` Create this Dockerfile. .debug[[intro/Copying_Files_During_Build.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Copying_Files_During_Build.md)] --- ## Testing our C program * Create `hello.c` and `Dockerfile` in the same direcotry. * Run `docker build -t hello .` in this directory. * Run `docker run hello`, you should see `Hello, world!`. Success! .debug[[intro/Copying_Files_During_Build.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Copying_Files_During_Build.md)] --- ## `COPY` and the build cache * Run the build again. * Now, modify `hello.c` and run the build again. * Docker can cache steps involving `COPY`. * Those steps will not be executed again if the files haven't been changed. .debug[[intro/Copying_Files_During_Build.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Copying_Files_During_Build.md)] --- ## Details * You can `COPY` whole directories recursively. * Older Dockerfiles also have the `ADD` instruction. <br/>It is similar but can automatically extract archives. * If we really wanted to compile C code in a compiler, we would: * Place it in a different directory, with the `WORKDIR` instruction. * Even better, use the `gcc` official image. .debug[[intro/Copying_Files_During_Build.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Copying_Files_During_Build.md)] --- class: pic .interstitial[] --- name: toc-multi-stage-builds class: title Multi-stage builds .nav[ [Previous section](#toc-copying-files-during-the-build) | [Back to table of contents](#toc-chapter-2) | [Next section](#toc-publishing-images-to-the-docker-hub) ] .debug[(automatically generated title slide)] --- # Multi-stage builds * In the previous example, our final image contain: * our `hello` program * its source code * the compiler * Only the first one is strictly necessary. * We are going to see how to obtain an image without the superfluous components. .debug[[intro/Multi_Stage_Builds.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Multi_Stage_Builds.md)] --- ## Multi-stage builds principles * At any point in our `Dockerfile`, we can add a new `FROM` line. * This line starts a new stage of our build. * Each stage can access the files of the previous stages with `COPY --from=...`. * When a build is tagged (with `docker build -t ...`), the last stage is tagged. * Previous stages are not discarded: they will be used for caching, and can be referenced. .debug[[intro/Multi_Stage_Builds.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Multi_Stage_Builds.md)] --- ## Multi-stage builds in practice * Each stage is numbered, starting at `0` * We can copy a file from a previous stage by indicating its number, e.g.: ```dockerfile COPY --from=0 /file/from/first/stage /location/in/current/stage ``` * We can also name stages, and reference these names: ```dockerfile FROM golang AS builder RUN ... FROM alpine COPY --from=builder /go/bin/mylittlebinary /usr/local/bin/ ``` .debug[[intro/Multi_Stage_Builds.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Multi_Stage_Builds.md)] --- ## Multi-stage builds for our C program We will change our Dockerfile to: * give a nickname to the first stage: `compiler` * add a second stage using the same `ubuntu` base image * add the `hello` binary to the second stage * make sure that `CMD` is in the second stage The resulting Dockerfile is on the next slide. .debug[[intro/Multi_Stage_Builds.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Multi_Stage_Builds.md)] --- ## Multi-stage build `Dockerfile` Here is the final Dockerfile: ```dockerfile FROM ubuntu AS compiler RUN apt-get update RUN apt-get install -y build-essential COPY hello.c / RUN make hello FROM ubuntu COPY --from=compiler /hello /hello CMD /hello ``` Let's build it, and check that it works correctly: ```bash docker build -t hellomultistage . docker run hellomultistage ``` .debug[[intro/Multi_Stage_Builds.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Multi_Stage_Builds.md)] --- ## Comparing single/multi-stage build image sizes List our images with `docker images`, and check the size of: - the `ubuntu` base image, - the single-stage `hello` image, - the multi-stage `hellomultistage` image. We can achieve even smaller images if we use smaller base images. However, if we use common base images (e.g. if we standardize on `ubuntu`), these common images will be pulled only once per node, so they are virtually "free." .debug[[intro/Multi_Stage_Builds.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Multi_Stage_Builds.md)] --- class: pic .interstitial[] --- name: toc-publishing-images-to-the-docker-hub class: title Publishing images to the Docker Hub .nav[ [Previous section](#toc-multi-stage-builds) | [Back to table of contents](#toc-chapter-2) | [Next section](#toc-tips-for-efficient-dockerfiles) ] .debug[(automatically generated title slide)] --- # Publishing images to the Docker Hub We have built our first images. We can now publish it to the Docker Hub! *You don't have to do the exercises in this section, because they require an account on the Docker Hub, and we don't want to force anyone to create one.* *Note, however, that creating an account on the Docker Hub is free (and doesn't require a credit card), and hosting public images is free as well.* .debug[[intro/Publishing_To_Docker_Hub.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Publishing_To_Docker_Hub.md)] --- ## Logging into our Docker Hub account * This can be done from the Docker CLI: ```bash docker login ``` .warning[When running Docker4Mac, Docker4Windows, or Docker on a Linux workstation, it can (and will when possible) integrate with your system's keyring to store your credentials securely. However, on most Linux servers, it will store your credentials in `~/.docker/config`.] .debug[[intro/Publishing_To_Docker_Hub.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Publishing_To_Docker_Hub.md)] --- ## Image tags and registry addresses * Docker images tags are like Git tags and branches. * They are like *bookmarks* pointing at a specific image ID. * Tagging an image doesn't *rename* an image: it adds another tag. * When pushing an image to a registry, the registry address is in the tag. Example: `registry.example.net:5000/image` * What about Docker Hub images? -- * `jpetazzo/clock` is, in fact, `index.docker.io/jpetazzo/clock` * `ubuntu` is, in fact, `library/ubuntu`, i.e. `index.docker.io/library/ubuntu` .debug[[intro/Publishing_To_Docker_Hub.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Publishing_To_Docker_Hub.md)] --- ## Tagging an image to push it on the Hub * Let's tag our `figlet` image (or any other to our liking): ```bash docker tag figlet jpetazzo/figlet ``` * And push it to the Hub: ```bash docker push jpetazzo/figlet ``` * That's it! -- * Anybody can now `docker run jpetazzo/figlet` anywhere. .debug[[intro/Publishing_To_Docker_Hub.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Publishing_To_Docker_Hub.md)] --- ## The goodness of automated builds * You can link a Docker Hub repository with a GitHub or BitBucket repository * Each push to GitHub or BitBucket will trigger a build on Docker Hub * If the build succeeds, the new image is available on Docker Hub * You can map tags and branches between source and container images * If you work with public repositories, this is free .debug[[intro/Publishing_To_Docker_Hub.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Publishing_To_Docker_Hub.md)] --- class: extra-details ## Setting up an automated build * We need a Dockerized repository! * Let's go to https://github.com/jpetazzo/trainingwheels and fork it. * Go to the Docker Hub (https://hub.docker.com/). * Select "Create" in the top-right bar, and select "Create Automated Build." * Connect your Docker Hub account to your GitHub account. * Select your user and the repository that we just forked. * Create. * Then go to "Build Settings." * Put `/www` in "Dockerfile Location" (or whichever directory the Dockerfile is in). * Click "Trigger" to build the repository immediately (without waiting for a git push). * Subsequent builds will happen automatically, thanks to GitHub hooks. .debug[[intro/Publishing_To_Docker_Hub.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Publishing_To_Docker_Hub.md)] --- class: pic .interstitial[] --- name: toc-tips-for-efficient-dockerfiles class: title Tips for efficient Dockerfiles .nav[ [Previous section](#toc-publishing-images-to-the-docker-hub) | [Back to table of contents](#toc-chapter-2) | [Next section](#toc-naming-and-inspecting-containers) ] .debug[(automatically generated title slide)] --- # Tips for efficient Dockerfiles We will see how to: * Reduce the number of layers. * Leverage the build cache so that builds can be faster. * Embed unit testing in the build process. .debug[[intro/Dockerfile_Tips.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Dockerfile_Tips.md)] --- ## Reducing the number of layers * Each line in a `Dockerfile` creates a new layer. * Build your `Dockerfile` to take advantage of Docker's caching system. * Combine commands by using `&&` to continue commands and `\` to wrap lines. Note: it is frequent to build a Dockerfile line by line: ```dockerfile RUN apt-get install thisthing RUN apt-get install andthatthing andthatotherone RUN apt-get install somemorestuff ``` And then refactor it trivially before shipping: ```dockerfile RUN apt-get install thisthing andthatthing andthatotherone somemorestuff ``` .debug[[intro/Dockerfile_Tips.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Dockerfile_Tips.md)] --- ## Avoid re-installing dependencies at each build * Classic Dockerfile problem: "each time I change a line of code, all my dependencies are re-installed!" * Solution: `COPY` dependency lists (`package.json`, `requirements.txt`, etc.) by themselves to avoid reinstalling unchanged dependencies every time. .debug[[intro/Dockerfile_Tips.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Dockerfile_Tips.md)] --- ## Example "bad" `Dockerfile` The dependencies are reinstalled every time, because the build system does not know if `requirements.txt` has been updated. ```bash FROM python MAINTAINER Docker Education Team <education@docker.com> COPY . /src/ WORKDIR /src RUN pip install -qr requirements.txt EXPOSE 5000 CMD ["python", "app.py"] ``` .debug[[intro/Dockerfile_Tips.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Dockerfile_Tips.md)] --- ## Fixed `Dockerfile` Adding the dependencies as a separate step means that Docker can cache more efficiently and only install them when `requirements.txt` changes. ```bash FROM python MAINTAINER Docker Education Team <education@docker.com> COPY ./requirements.txt /tmp/requirements.txt RUN pip install -qr /tmp/requirements.txt COPY . /src/ WORKDIR /src EXPOSE 5000 CMD ["python", "app.py"] ``` .debug[[intro/Dockerfile_Tips.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Dockerfile_Tips.md)] --- ## Embedding unit tests in the build process ```dockerfile FROM <baseimage> RUN <install dependencies> COPY <code> RUN <build code> RUN <install test dependencies> COPY <test data sets and fixtures> RUN <unit tests> FROM <baseimage> RUN <install dependencies> COPY <vcode> RUN <build code> CMD, EXPOSE ... ``` * The build fails as soon as an instructions fails * If `RUN <unit tests>` fails, the build doesn't produce an image * If it succeeds, it produces a clean image (without test libraries and data) .debug[[intro/Dockerfile_Tips.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Dockerfile_Tips.md)] --- class: pic .interstitial[] --- name: toc-naming-and-inspecting-containers class: title Naming and inspecting containers .nav[ [Previous section](#toc-tips-for-efficient-dockerfiles) | [Back to table of contents](#toc-chapter-3) | [Next section](#toc-container-networking-basics) ] .debug[(automatically generated title slide)] --- class: title # Naming and inspecting containers  .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- ## Objectives In this lesson, we will learn about an important Docker concept: container *naming*. Naming allows us to: * Reference easily a container. * Ensure unicity of a specific container. We will also see the `inspect` command, which gives a lot of details about a container. .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- ## Naming our containers So far, we have referenced containers with their ID. We have copy-pasted the ID, or used a shortened prefix. But each container can also be referenced by its name. If a container is named `thumbnail-worker`, I can do: ```bash $ docker logs thumbnail-worker $ docker stop thumbnail-worker etc. ``` .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- ## Default names When we create a container, if we don't give a specific name, Docker will pick one for us. It will be the concatenation of: * A mood (furious, goofy, suspicious, boring...) * The name of a famous inventor (tesla, darwin, wozniak...) Examples: `happy_curie`, `clever_hopper`, `jovial_lovelace` ... .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- ## Specifying a name You can set the name of the container when you create it. ```bash $ docker run --name ticktock jpetazzo/clock ``` If you specify a name that already exists, Docker will refuse to create the container. This lets us enforce unicity of a given resource. .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- ## Renaming containers * You can rename containers with `docker rename`. * This allows you to "free up" a name without destroying the associated container. .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- ## Inspecting a container The `docker inspect` command will output a very detailed JSON map. ```bash $ docker inspect <containerID> [{ ... (many pages of JSON here) ... ``` There are multiple ways to consume that information. .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- ## Parsing JSON with the Shell * You *could* grep and cut or awk the output of `docker inspect`. * Please, don't. * It's painful. * If you really must parse JSON from the Shell, use JQ! (It's great.) ```bash $ docker inspect <containerID> | jq . ``` * We will see a better solution which doesn't require extra tools. .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- ## Using `--format` You can specify a format string, which will be parsed by Go's text/template package. ```bash $ docker inspect --format '{{ json .Created }}' <containerID> "2015-02-24T07:21:11.712240394Z" ``` * The generic syntax is to wrap the expression with double curly braces. * The expression starts with a dot representing the JSON object. * Then each field or member can be accessed in dotted notation syntax. * The optional `json` keyword asks for valid JSON output. <br/>(e.g. here it adds the surrounding double-quotes.) .debug[[intro/Naming_And_Inspecting.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Naming_And_Inspecting.md)] --- class: pic .interstitial[] --- name: toc-container-networking-basics class: title Container networking basics .nav[ [Previous section](#toc-naming-and-inspecting-containers) | [Back to table of contents](#toc-chapter-3) | [Next section](#toc-container-network-drivers) ] .debug[(automatically generated title slide)] --- class: title # Container networking basics  .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Objectives We will now run network services (accepting requests) in containers. At the end of this section, you will be able to: * Run a network service in a container. * Manipulate container networking basics. * Find a container's IP address. We will also explain the different network models used by Docker. .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## A simple, static web server Run the Docker Hub image `nginx`, which contains a basic web server: ```bash $ docker run -d -P nginx 66b1ce719198711292c8f34f84a7b68c3876cf9f67015e752b94e189d35a204e ``` * Docker will download the image from the Docker Hub. * `-d` tells Docker to run the image in the background. * `-P` tells Docker to make this service reachable from other computers. <br/>(`-P` is the short version of `--publish-all`.) But, how do we connect to our web server now? .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Finding our web server port We will use `docker ps`: ```bash $ docker ps CONTAINER ID IMAGE ... PORTS ... e40ffb406c9e nginx ... 0.0.0.0:32769->80/tcp, 0.0.0.0:32768->443/tcp ... ``` * The web server is running on ports 80 and 443 inside the container. * Those ports are mapped to ports 32769 and 32768 on our Docker host. We will explain the whys and hows of this port mapping. But first, let's make sure that everything works properly. .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Connecting to our web server (GUI) Point your browser to the IP address of your Docker host, on the port shown by `docker ps` for container port 80.  .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Connecting to our web server (CLI) You can also use `curl` directly from the Docker host. Make sure to use the right port number if it is different from the example below: ```bash $ curl localhost:32769 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> ... ``` .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Why are we mapping ports? * We are out of IPv4 addresses. * Containers cannot have public IPv4 addresses. * They have private addresses. * Services have to be exposed port by port. * Ports have to be mapped to avoid conflicts. .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Finding the web server port in a script Parsing the output of `docker ps` would be painful. There is a command to help us: ```bash $ docker port <containerID> 80 32769 ``` .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Manual allocation of port numbers If you want to set port numbers yourself, no problem: ```bash $ docker run -d -p 80:80 nginx $ docker run -d -p 8000:80 nginx $ docker run -d -p 8080:80 -p 8888:80 nginx ``` * We are running two NGINX web servers. * The first one is exposed on port 80. * The second one is exposed on port 8000. * The third one is exposed on ports 8080 and 8888. Note: the convention is `port-on-host:port-on-container`. .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Plumbing containers into your infrastructure There are many ways to integrate containers in your network. * Start the container, letting Docker allocate a public port for it. <br/>Then retrieve that port number and feed it to your configuration. * Pick a fixed port number in advance, when you generate your configuration. <br/>Then start your container by setting the port numbers manually. * Use a network plugin, connecting your containers with e.g. VLANs, tunnels... * Enable *Swarm Mode* to deploy across a cluster. <br/>The container will then be reachable through any node of the cluster. When using Docker through an extra management layer like Mesos or Kubernetes, these will usually provide their own mechanism to expose containers. .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Finding the container's IP address We can use the `docker inspect` command to find the IP address of the container. ```bash $ docker inspect --format '{{ .NetworkSettings.IPAddress }}' <yourContainerID> 172.17.0.3 ``` * `docker inspect` is an advanced command, that can retrieve a ton of information about our containers. * Here, we provide it with a format string to extract exactly the private IP address of the container. .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Pinging our container We can test connectivity to the container using the IP address we've just discovered. Let's see this now by using the `ping` tool. ```bash $ ping <ipAddress> 64 bytes from <ipAddress>: icmp_req=1 ttl=64 time=0.085 ms 64 bytes from <ipAddress>: icmp_req=2 ttl=64 time=0.085 ms 64 bytes from <ipAddress>: icmp_req=3 ttl=64 time=0.085 ms ``` .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- ## Section summary We've learned how to: * Expose a network port. * Manipulate container networking basics. * Find a container's IP address. In the next chapter, we will see how to connect containers together without exposing their ports. .debug[[intro/Container_Networking_Basics.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Networking_Basics.md)] --- class: pic .interstitial[] --- name: toc-container-network-drivers class: title Container network drivers .nav[ [Previous section](#toc-container-networking-basics) | [Back to table of contents](#toc-chapter-3) | [Next section](#toc-the-container-network-model) ] .debug[(automatically generated title slide)] --- # Container network drivers The Docker Engine supports many different network drivers. The built-in drivers include: * `bridge` (default) * `none` * `host` * `container` The driver is selected with `docker run --net ...`. The different drivers are explained with more details on the following slides. .debug[[intro/Network_Drivers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Network_Drivers.md)] --- ## The default bridge * By default, the container gets a virtual `eth0` interface. <br/>(In addition to its own private `lo` loopback interface.) * That interface is provided by a `veth` pair. * It is connected to the Docker bridge. <br/>(Named `docker0` by default; configurable with `--bridge`.) * Addresses are allocated on a private, internal subnet. <br/>(Docker uses 172.17.0.0/16 by default; configurable with `--bip`.) * Outbound traffic goes through an iptables MASQUERADE rule. * Inbound traffic goes through an iptables DNAT rule. * The container can have its own routes, iptables rules, etc. .debug[[intro/Network_Drivers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Network_Drivers.md)] --- ## The null driver * Container is started with `docker run --net none ...` * It only gets the `lo` loopback interface. No `eth0`. * It can't send or receive network traffic. * Useful for isolated/untrusted workloads. .debug[[intro/Network_Drivers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Network_Drivers.md)] --- ## The host driver * Container is started with `docker run --net host ...` * It sees (and can access) the network interfaces of the host. * It can bind any address, any port (for ill and for good). * Network traffic doesn't have to go through NAT, bridge, or veth. * Performance = native! Use cases: * Performance sensitive applications (VOIP, gaming, streaming...) * Peer discovery (e.g. Erlang port mapper, Raft, Serf...) .debug[[intro/Network_Drivers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Network_Drivers.md)] --- ## The container driver * Container is started with `docker run --net container:id ...` * It re-uses the network stack of another container. * It shares with this other container the same interfaces, IP address(es), routes, iptables rules, etc. * Those containers can communicate over their `lo` interface. <br/>(i.e. one can bind to 127.0.0.1 and the others can connect to it.) .debug[[intro/Network_Drivers.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Network_Drivers.md)] --- class: pic .interstitial[] --- name: toc-the-container-network-model class: title The Container Network Model .nav[ [Previous section](#toc-container-network-drivers) | [Back to table of contents](#toc-chapter-3) | [Next section](#toc-service-discovery-with-containers) ] .debug[(automatically generated title slide)] --- class: title # The Container Network Model  .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Objectives We will learn about the CNM (Container Network Model). At the end of this lesson, you will be able to: * Create a private network for a group of containers. * Use container naming to connect services together. * Dynamically connect and disconnect containers to networks. * Set the IP address of a container. We will also explain the principle of overlay networks and network plugins. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## The Container Network Model The CNM was introduced in Engine 1.9.0 (November 2015). The CNM adds the notion of a *network*, and a new top-level command to manipulate and see those networks: `docker network`. ```bash $ docker network ls NETWORK ID NAME DRIVER 6bde79dfcf70 bridge bridge 8d9c78725538 none null eb0eeab782f4 host host 4c1ff84d6d3f blog-dev overlay 228a4355d548 blog-prod overlay ``` .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## What's in a network? * Conceptually, a network is a virtual switch. * It can be local (to a single Engine) or global (spanning multiple hosts). * A network has an IP subnet associated to it. * Docker will allocate IP addresses to the containers connected to a network. * Containers can be connected to multiple networks. * Containers can be given per-network names and aliases. * The names and aliases can be resolved via an embedded DNS server. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Network implementation details * A network is managed by a *driver*. * All the drivers that we have seen before are available. * A new multi-host driver, *overlay*, is available out of the box. * More drivers can be provided by plugins (OVS, VLAN...) * A network can have a custom IPAM (IP allocator). .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Differences with the CNI * CNI = Container Network Interface * CNI is used notably by Kubernetes * With CNI, all the nodes and containers are on a single IP network * Both CNI and CNM offer the same functionality, but with very different methods .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Creating a network Let's create a network called `dev`. ```bash $ docker network create dev 4c1ff84d6d3f1733d3e233ee039cac276f425a9d5228a4355d54878293a889ba ``` The network is now visible with the `network ls` command: ```bash $ docker network ls NETWORK ID NAME DRIVER 6bde79dfcf70 bridge bridge 8d9c78725538 none null eb0eeab782f4 host host 4c1ff84d6d3f dev bridge ``` .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Placing containers on a network We will create a *named* container on this network. It will be reachable with its name, `es`. ```bash $ docker run -d --name es --net dev elasticsearch:2 8abb80e229ce8926c7223beb69699f5f34d6f1d438bfc5682db893e798046863 ``` .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Communication between containers Now, create another container on this network. .small[ ```bash $ docker run -ti --net dev alpine sh root@0ecccdfa45ef:/# ``` ] From this new container, we can resolve and ping the other one, using its assigned name: .small[ ```bash / # ping es PING es (172.18.0.2) 56(84) bytes of data. 64 bytes from es.dev (172.18.0.2): icmp_seq=1 ttl=64 time=0.221 ms 64 bytes from es.dev (172.18.0.2): icmp_seq=2 ttl=64 time=0.114 ms 64 bytes from es.dev (172.18.0.2): icmp_seq=3 ttl=64 time=0.114 ms ^C --- es ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.114/0.149/0.221/0.052 ms root@0ecccdfa45ef:/# ``` ] .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Resolving container addresses In Docker Engine 1.9, name resolution is implemented with `/etc/hosts`, and updating it each time containers are added/removed. .small[ ```bash [root@0ecccdfa45ef /]# cat /etc/hosts 172.18.0.3 0ecccdfa45ef 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 172.18.0.2 es 172.18.0.2 es.dev ``` ] In Docker Engine 1.10, this has been replaced by a dynamic resolver. (This avoids race conditions when updating `/etc/hosts`.) .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: pic .interstitial[] --- name: toc-service-discovery-with-containers class: title Service discovery with containers .nav[ [Previous section](#toc-the-container-network-model) | [Back to table of contents](#toc-chapter-3) | [Next section](#toc-ambassadors) ] .debug[(automatically generated title slide)] --- # Service discovery with containers * Let's try to run an application that requires two containers. * The first container is a web server. * The other one is a redis data store. * We will place them both on the `dev` network created before. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Running the web server * The application is provided by the container image `jpetazzo/trainingwheels`. * We don't know much about it so we will try to run it and see what happens! Start the container, exposing all its ports: ```bash $ docker run --net dev -d -P jpetazzo/trainingwheels ``` Check the port that has been allocated to it: ```bash $ docker ps -l ``` .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Test the web server * If we connect to the application now, we will see an error page:  * This is because the Redis service is not running. * This container tries to resolve the name `redis`. Note: we're not using a FQDN or an IP address here; just `redis`. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Start the data store * We need to start a Redis container. * That container must be on the same network as the web server. * It must have the right name (`redis`) so the application can find it. Start the container: ```bash $ docker run --net dev --name redis -d redis ``` .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Test the web server again * If we connect to the application now, we should see that the app is working correctly:  * When the app tries to resolve `redis`, instead of getting a DNS error, it gets the IP address of our Redis container. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## A few words on *scope* * What if we want to run multiple copies of our application? * Since names are unique, there can be only one container named `redis` at a time. * However, we can specify the network name of our container with `--net-alias`. * `--net-alias` is scoped per network, and independent from the container name. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Using a network alias instead of a name Let's remove the `redis` container: ```bash $ docker rm -f redis ``` And create one that doesn't block the `redis` name: ```bash $ docker run --net dev --net-alias redis -d redis ``` Check that the app still works (but the counter is back to 1, since we wiped out the old Redis container). .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: x-extra-details ## Names are *local* to each network Let's try to ping our `es` container from another container, when that other container is *not* on the `dev` network. ```bash $ docker run --rm alpine ping es ping: bad address 'es' ``` Names can be resolved only when containers are on the same network. Containers can contact each other only when they are on the same network (you can try to ping using the IP address to verify). .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Network aliases We would like to have another network, `prod`, with its own `es` container. But there can be only one container named `es`! We will use *network aliases*. A container can have multiple network aliases. Network aliases are *local* to a given network (only exist in this network). Multiple containers can have the same network alias (even on the same network). In Docker Engine 1.11, resolving a network alias yields the IP addresses of all containers holding this alias. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Creating containers on another network Create the `prod` network. ```bash $ docker create network prod 5a41562fecf2d8f115bedc16865f7336232a04268bdf2bd816aecca01b68d50c ``` We can now create multiple containers with the `es` alias on the new `prod` network. ```bash $ docker run -d --name prod-es-1 --net-alias es --net prod elasticsearch:2 38079d21caf0c5533a391700d9e9e920724e89200083df73211081c8a356d771 $ docker run -d --name prod-es-2 --net-alias es --net prod elasticsearch:2 1820087a9c600f43159688050dcc164c298183e1d2e62d5694fd46b10ac3bc3d ``` .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Resolving network aliases Let's try DNS resolution first, using the `nslookup` tool that ships with the `alpine` image. ```bash $ docker run --net prod --rm alpine nslookup es Name: es Address 1: 172.23.0.3 prod-es-2.prod Address 2: 172.23.0.2 prod-es-1.prod ``` (You can ignore the `can't resolve '(null)'` errors.) .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Connecting to aliased containers Each ElasticSearch instance has a name (generated when it is started). This name can be seen when we issue a simple HTTP request on the ElasticSearch API endpoint. Try the following command a few times: .small[ ```bash $ docker run --rm --net dev centos curl -s es:9200 { "name" : "Tarot", ... } ``` ] Then try it a few times by replacing `--net dev` with `--net prod`: .small[ ```bash $ docker run --rm --net prod centos curl -s es:9200 { "name" : "The Symbiote", ... } ``` ] .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Good to know ... * Docker will not create network names and aliases on the default `bridge` network. * Therefore, if you want to use those features, you have to create a custom network first. * Network aliases are *not* unique on a given network. * i.e., multiple containers can have the same alias on the same network. * In that scenario, the Docker DNS server will return multiple records. <br/> (i.e. you will get DNS round robin out of the box.) * Enabling *Swarm Mode* gives access to clustering and load balancing with IPVS. * Creation of networks and network aliases is generally automated with tools like Compose. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## A few words about round robin DNS Don't rely exclusively on round robin DNS to achieve load balancing. Many factors can affect DNS resolution, and you might see: - all traffic going to a single instance; - traffic being split (unevenly) between some instances; - different behavior depending on your application language; - different behavior depending on your base distro; - different behavior depending on other factors (sic). It's OK to use DNS to discover available endpoints, but remember that you have to re-resolve every now and then to discover new endpoints. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Custom networks When creating a network, extra options can be provided. * `--internal` disables outbound traffic (the network won't have a default gateway). * `--gateway` indicates which address to use for the gateway (when utbound traffic is allowed). * `--subnet` (in CIDR notation) indicates the subnet to use. * `--ip-range` (in CIDR notation) indicates the subnet to allocate from. * `--aux-address` allows to specify a list of reserved addresses (which won't be allocated to containers). .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Setting containers' IP address * It is possible to set a container's address with `--ip`. * The IP address has to be within the subnet used for the container. A full example would look like this. ```bash $ docker network create --subnet 10.66.0.0/16 pubnet 42fb16ec412383db6289a3e39c3c0224f395d7f85bcb1859b279e7a564d4e135 $ docker run --net pubnet --ip 10.66.66.66 -d nginx b2887adeb5578a01fd9c55c435cad56bbbe802350711d2743691f95743680b09 ``` *Note: don't hard code container IP addresses in your code!* *I repeat: don't hard code container IP addresses in your code!* .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Overlay networks * The features we've seen so far only work when all containers are on a single host. * If containers span multiple hosts, we need an *overlay* network to connect them together. * Docker ships with a default network plugin, `overlay`, implementing an overlay network leveraging VXLAN. * Other plugins (Weave, Calico...) can provide overlay networks as well. * Once you have an overlay network, *all the features that we've used in this chapter work identically.* .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Multi-host networking (overlay) Out of the scope for this intro-level workshop! Very short instructions: - enable Swarm Mode (`docker swarm init` then `docker swarm join` on other nodes) - `docker network create mynet --driver overlay` - `docker service create --network mynet myimage` See http://jpetazzo.github.io/container.training for all the deets about clustering! .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: extra-details ## Multi-host networking (plugins) Out of the scope for this intro-level workshop! General idea: - install the plugin (they often ship within containers) - run the plugin (if it's in a container, it will often require extra parameters; don't just `docker run` it blindly!) - some plugins require configuration or activation (creating a special file that tells Docker "use the plugin whose control socket is at the following location") - you can then `docker network create --driver pluginname` .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- ## Section summary We've learned how to: * Create private networks for groups of containers. * Assign IP addresses to containers. * Use container naming to implement service discovery. .debug[[intro/Container_Network_Model.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Container_Network_Model.md)] --- class: pic .interstitial[] --- name: toc-ambassadors class: title Ambassadors .nav[ [Previous section](#toc-service-discovery-with-containers) | [Back to table of contents](#toc-chapter-3) | [Next section](#toc-local-development-workflow-with-docker) ] .debug[(automatically generated title slide)] --- class: title # Ambassadors  .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## The ambassador pattern Ambassadors are containers that "masquerade" or "proxy" for another service. They abstract the connection details for this services, and can help with: * discovery (where is my service actually running?) * migration (what if my service has to be moved while I use it?) * fail over (how do I know to which instance of a replicated service I should connect?) * load balancing (how to I spread my requests across multiple instances of a service?) * authentication (what if my service requires credentials, certificates, or otherwise?) .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## Introduction to Ambassadors The ambassador pattern: * Takes advantage of Docker's per-container naming system and abstracts connections between services. * Allows you to manage services without hard-coding connection information inside applications. To do this, instead of directly connecting containers you insert ambassador containers. .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] ---  .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## Interacting with ambassadors * The web container uses normal Docker networking to connect to the ambassador. * The database container also talks with an ambassador. * For both containers, the ambassador is totally transparent. <br/> (There is no difference between normal operation and operation with an ambassador.) * If the database container is moved (or a failover happens), its new location will be tracked by the ambassador containers, and the web application container will still be able to connect, without reconfiguration. .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## Ambassadors for simple service discovery Use case: * my application code connects to `redis` on the default port (6379), * my Redis service runs on another machine, on a non-default port (e.g. 12345), * I want to use an ambassador to let my application connect without modification. The ambassador will be: * a container running right next to my application, * using the name `redis` (or linked as `redis`), * listening on port 6379, * forwarding connections to the actual Redis service. .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## Ambassadors for service migration Use case: * my application code still connects to `redis`, * my Redis service runs somewhere else, * my Redis service is moved to a different host+port, * the location of the Redis service is given to me via e.g. DNS SRV records, * I want to use an ambassador to automatically connect to the new location, with as little disruption as possible. The ambassador will be: * the same kind of container as before, * running an additional routine to monitor DNS SRV records, * updating the forwarding destination when the DNS SRV records are updated. .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## Ambassadors for credentials injection Use case: * my application code still connects to `redis`, * my application code doesn't provide Redis credentials, * my production Redis service requires credentials, * my staging Redis service requires different credentials, * I want to use an ambassador to abstract those credentials. The ambassador will be: * a container using the name `redis` (or a link), * passed the credentials to use, * running a custom proxy that accepts connections on Redis default port, * performing authentication with the target Redis service before forwarding traffic. .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## Ambassadors for load balancing Use case: * my application code connects to a web service called `api`, * I want to run multiple instances of the `api` backend, * those instances will be on different machines and ports, * I want to use an ambassador to abstract those details. The ambassador will be: * a container using the name `api` (or a link), * passed the list of backends to use (statically or dynamically), * running a load balancer (e.g. HAProxy or NGINX), * dispatching requests across all backends transparently. .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## "Ambassador" is a *pattern* There are many ways to implement the pattern. Different deployments will use different underlying technologies. * On-premise deployments with a trusted network can track container locations in e.g. Zookeeper, and generate HAproxy configurations each time a location key changes. * Public cloud deployments or deployments across unsafe networks can add TLS encryption. * Ad-hoc deployments can use a master-less discovery protocol like avahi to register and discover services. * It is also possible to do one-shot reconfiguration of the ambassadors. It is slightly less dynamic but has much less requirements. * Ambassadors can be used in addition to, or instead of, overlay networks. .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- ## Section summary We've learned how to: * Understand the ambassador pattern and what it is used for (service portability). For more information about the ambassador pattern, including demos on Swarm and ECS: * AWS re:invent 2015 [DVO317](https://www.youtube.com/watch?v=7CZFpHUPqXw) * [SwarmWeek video about Swarm+Compose](https://youtube.com/watch?v=qbIvUvwa6As) .debug[[intro/Ambassadors.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Ambassadors.md)] --- class: pic .interstitial[] --- name: toc-local-development-workflow-with-docker class: title Local development workflow with Docker .nav[ [Previous section](#toc-ambassadors) | [Back to table of contents](#toc-chapter-4) | [Next section](#toc-working-with-volumes) ] .debug[(automatically generated title slide)] --- class: title # Local development workflow with Docker  .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Objectives At the end of this section, you will be able to: * Share code between container and host. * Use a simple local development workflow. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Containerized local development environments We want to solve the following issues: - "Works on my machine" - "Not the same version" - "Missing dependency" By using Docker containers, we will get a consistent development environment. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Working on the "namer" application * We have to work on some application whose code is at: https://github.com/jpetazzo/namer. * What is it? We don't know yet! * Let's download the code. ```bash $ git clone https://github.com/jpetazzo/namer ``` .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Looking at the code ```bash $ cd namer $ ls -1 company_name_generator.rb config.ru docker-compose.yml Dockerfile Gemfile ``` -- Aha, a `Gemfile`! This is Ruby. Probably. We know this. Maybe? .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Looking at the `Dockerfile` ```dockerfile FROM ruby MAINTAINER Education Team at Docker <education@docker.com> COPY . /src WORKDIR /src RUN bundler install CMD ["rackup", "--host", "0.0.0.0"] EXPOSE 9292 ``` * This application is using a base `ruby` image. * The code is copied in `/src`. * Dependencies are installed with `bundler`. * The application is started with `rackup`. * It is listening on port 9292. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Building and running the "namer" application * Let's build the application with the `Dockerfile`! -- ```bash $ docker build -t namer . ``` -- * Then run it. *We need to expose its ports.* -- ```bash $ docker run -dP namer ``` -- * Check on which port the container is listening. -- ```bash $ docker ps -l ``` .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Connecting to our application * Point our browser to our Docker node, on the port allocated to the container. -- * Hit "reload" a few times. -- * This is an enterprise-class, carrier-grade, ISO-compliant company name generator! (With 50% more bullshit than the average competition!) (Wait, was that 50% more, or 50% less? *Anyway!*)  .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Making changes to the code Option 1: * Edit the code locally * Rebuild the image * Re-run the container Option 2: * Enter the container (with `docker exec`) * Install an editor * Make changes from within the container Option 3: * Use a *volume* to mount local files into the container * Make changes locally * Changes are reflected into the container .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Our first volume We will tell Docker to map the current directory to `/src` in the container. ```bash $ docker run -d -v $(pwd):/src -P namer ``` * `-d`: the container should run in detached mode (in the background). * `-v`: the following host directory should be mounted inside the container. * `-P`: publish all the ports exposed by this image. * `namer` is the name of the image we will run. * We don't specify a command to run because is is already set in the Dockerfile. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Mounting volumes inside containers The `-v` flag mounts a directory from your host into your Docker container. The flag structure is: ```bash [host-path]:[container-path]:[rw|ro] ``` * If `[host-path]` or `[container-path]` doesn't exist it is created. * You can control the write status of the volume with the `ro` and `rw` options. * If you don't specify `rw` or `ro`, it will be `rw` by default. There will be a full chapter about volumes! .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Testing the development container * Check the port used by our new container. ```bash $ docker ps -l CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 045885b68bc5 namer rackup 3 seconds ago Up ... 0.0.0.0:32770->9292/tcp ... ``` * Open the application in your web browser. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Making a change to our application Our customer really doesn't like the color of our text. Let's change it. ```bash $ vi company_name_generator.rb ``` And change ```css color: royalblue; ``` To: ```css color: red; ``` .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Viewing our changes * Reload the application in our browser. -- * The color should have changed.  .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Understanding volumes * Volumes are *not* copying or synchronizing files between the host and the container. * Volumes are *bind mounts*: a kernel mechanism associating a path to another. * Bind mounts are *kind of* similar to symbolic links, but at a very different level. * Changes made on the host or on the container will be visible on the other side. (Since under the hood, it's the same file on both anyway.) .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Trash your servers and burn your code *(This is the title of a [2013 blog post](http://chadfowler.com/2013/06/23/immutable-deployments.html) by Chad Fowler, where he explains the concept of immutable infrastructure.)* -- * Let's mess up majorly with our container. (Remove files or whatever.) * Now, how can we fix this? -- * Our old container (with the blue version of the code) is still running. * See on which port it is exposed: ```bash docker ps ``` * Point our browser to it to confirm that it still works fine. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Immutable infrastructure in a nutshell * Instead of *updating* a server, we deploy a new one. * This might be challenging with classical servers, but it's trivial with containers. * In fact, with Docker, the most logical workflow is to build a new image and run it. * If something goes wrong with the new image, we can always restart the old one. * We can even keep both versions running side by side. If this pattern sounds interesting, you might want to read about *blue/green deployment* and *canary deployments*. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Improving the workflow The workflow that we showed is nice, but it requires us to: * keep track of all the `docker run` flags required to run the container, * inspect the `Dockerfile` to know which path(s) to mount, * write scripts to hide that complexity. There has to be a better way! .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Docker Compose to the rescue * Docker Compose allows us to "encode" `docker run` parameters in a YAML file. * Here is the `docker-compose.yml` file that we can use for our "namer" app: ```yaml www: build: . volumes: - .:/src ports: - 80:9292 ``` * Try it: ```bash $ docker-compose up -d ``` .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Working with Docker Compose * When you see a `docker-compose.yml` file, you can use `docker-compose up`. * It can build images and run them with the required parameters. * Compose can also deal with complex, multi-container apps. (More on this later!) .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Recap of the development workflow 1. Write a Dockerfile to build an image containing our development environment. <br/> (Rails, Django, ... and all the dependencies for our app) 2. Start a container from that image. <br/> Use the `-v` flag to mount our source code inside the container. 3. Edit the source code outside the containers, using regular tools. <br/> (vim, emacs, textmate...) 4. Test the application. <br/> (Some frameworks pick up changes automatically. <br/>Others require you to Ctrl-C + restart after each modification.) 5. Iterate and repeat steps 3 and 4 until satisfied. 6. When done, commit+push source code changes. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- class: extra-details ## Debugging inside the container Docker has a command called `docker exec`. It allows users to run a new process in a container which is already running. If sometimes you find yourself wishing you could SSH into a container: you can use `docker exec` instead. You can get a shell prompt inside an existing container this way, or run an arbitrary process for automation. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- class: extra-details ## `docker exec` example ```bash $ # You can run ruby commands in the area the app is running and more! $ docker exec -it <yourContainerId> bash root@5ca27cf74c2e:/opt/namer# irb irb(main):001:0> [0, 1, 2, 3, 4].map {|x| x ** 2}.compact => [0, 1, 4, 9, 16] irb(main):002:0> exit ``` .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- class: extra-details ## Stopping the container Now that we're done let's stop our container. ```bash $ docker stop <yourContainerID> ``` And remove it. ```bash $ docker rm <yourContainerID> ``` .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- ## Section summary We've learned how to: * Share code between container and host. * Set our working directory. * Use a simple local development workflow. .debug[[intro/Local_Development_Workflow.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Local_Development_Workflow.md)] --- class: pic .interstitial[] --- name: toc-working-with-volumes class: title Working with volumes .nav[ [Previous section](#toc-local-development-workflow-with-docker) | [Back to table of contents](#toc-chapter-4) | [Next section](#toc-compose-for-development-stacks) ] .debug[(automatically generated title slide)] --- class: title # Working with volumes  .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Objectives At the end of this section, you will be able to: * Create containers holding volumes. * Share volumes across containers. * Share a host directory with one or many containers. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Working with volumes Docker volumes can be used to achieve many things, including: * Bypassing the copy-on-write system to obtain native disk I/O performance. * Bypassing copy-on-write to leave some files out of `docker commit`. * Sharing a directory between multiple containers. * Sharing a directory between the host and a container. * Sharing a *single file* between the host and a container. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Volumes are special directories in a container Volumes can be declared in two different ways. * Within a `Dockerfile`, with a `VOLUME` instruction. ```dockerfile VOLUME /uploads ``` * On the command-line, with the `-v` flag for `docker run`. ```bash $ docker run -d -v /uploads myapp ``` In both cases, `/uploads` (inside the container) will be a volume. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Volumes bypass the copy-on-write system Volumes act as passthroughs to the host filesystem. * The I/O performance on a volume is exactly the same as I/O performance on the Docker host. * When you `docker commit`, the content of volumes is not brought into the resulting image. * If a `RUN` instruction in a `Dockerfile` changes the content of a volume, those changes are not recorded neither. * If a container is started with the `--read-only` flag, the volume will still be writable (unless the volume is a read-only volume). .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Volumes can be shared across containers You can start a container with *exactly the same volumes* as another one. The new container will have the same volumes, in the same directories. They will contain exactly the same thing, and remain in sync. Under the hood, they are actually the same directories on the host anyway. This is done using the `--volumes-from` flag for `docker run`. We will see an example in the following slides. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Sharing app server logs with another container Let's start a Tomcat container: ```bash $ docker run --name webapp -d -p 8080:8080 -v /usr/local/tomcat/logs ``` Now, start an `alpine` container accessing the same volume: ```bash $ docker run --volumes-from webapp alpine sh -c "tail -f /usr/local/tomcat/logs/*" ``` Then, from another window, send requests to our Tomcat container: ```bash $ curl localhost:8080 ``` .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Volumes exist independently of containers If a container is stopped, its volumes still exist and are available. Volumes can be listed and manipulated with `docker volume` subcommands: ```bash $ docker volume ls DRIVER VOLUME NAME local 5b0b65e4316da67c2d471086640e6005ca2264f3... local pgdata-prod local pgdata-dev local 13b59c9936d78d109d094693446e174e5480d973... ``` Some of those volume names were explicit (pgdata-prod, pgdata-dev). The others (the hex IDs) were generated automatically by Docker. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Naming volumes * Volumes can be created without a container, then used in multiple containers. Let's create a couple of volumes directly. ```bash $ docker volume create webapps webapps ``` ```bash $ docker volume create logs logs ``` Volumes are not anchored to a specific path. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Using our named volumes * Volumes are used with the `-v` option. * When a host path does not contain a /, it is considered to be a volume name. Let's start a web server using the two previous volumes. ```bash $ docker run -d -p 1234:8080 \ -v logs:/usr/local/tomcat/logs \ -v webapps:/usr/local/tomcat/webapps \ tomcat ``` Check that it's running correctly: ```bash $ curl localhost:1234 ... (Tomcat tells us how happy it is to be up and running) ... ``` .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Using a volume in another container * We will make changes to the volume from another container. * In this example, we will run a text editor in the other container. (But this could be a FTP server, a WebDAV server, a Git receiver...) Let's start another container using the `webapps` volume. ```bash $ docker run -v webapps:/webapps -w /webapps -ti alpine vi ROOT/index.jsp ``` Vandalize the page, save, exit. Then run `curl localhost:1234` again to see your changes. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Managing volumes explicitly In some cases, you want a specific directory on the host to be mapped inside the container: * You want to manage storage and snapshots yourself. (With LVM, or a SAN, or ZFS, or anything else!) * You have a separate disk with better performance (SSD) or resiliency (EBS) than the system disk, and you want to put important data on that disk. * You want to share your source directory between your host (where the source gets edited) and the container (where it is compiled or executed). Wait, we already met the last use-case in our example development workflow! Nice. ```bash $ docker run -d -v /path/on/the/host:/path/in/container image ... ``` .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Migrating data with `--volumes-from` The `--volumes-from` option tells Docker to re-use all the volumes of an existing container. * Scenario: migrating from Redis 2.8 to Redis 3.0. * We have a container (`myredis`) running Redis 2.8. * Stop the `myredis` container. * Start a new container, using the Redis 3.0 image, and the `--volumes-from` option. * The new container will inherit the data of the old one. * Newer containers can use `--volumes-from` too. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Data migration in practice Let's create a Redis container. ```bash $ docker run -d --name redis28 redis:2.8 ``` Connect to the Redis container and set some data. ```bash $ docker run -ti --link redis28:redis alpine telnet redis 6379 ``` Issue the following commands: ```bash SET counter 42 INFO server SAVE QUIT ``` .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Upgrading Redis Stop the Redis container. ```bash $ docker stop redis28 ``` Start the new Redis container. ```bash $ docker run -d --name redis30 --volumes-from redis28 redis:3.0 ``` .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Testing the new Redis Connect to the Redis container and see our data. ```bash docker run -ti --link redis30:redis alpine telnet redis 6379 ``` Issue a few commands. ```bash GET counter INFO server QUIT ``` .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Volumes lifecycle * When you remove a container, its volumes are kept around. * You can list them with `docker volume ls`. * You can access them by creating a container with `docker run -v`. * You can remove them with `docker volume rm` or `docker system prune`. Ultimately, _you_ are the one responsible for logging, monitoring, and backup of your volumes. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Checking volumes defined by an image Wondering if an image has volumes? Just use `docker inspect`: ```bash $ # docker inspect training/datavol [{ "config": { . . . "Volumes": { "/var/webapp": {} }, . . . }] ``` .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: extra-details ## Checking volumes used by a container To look which paths are actually volumes, and to what they are bound, use `docker inspect` (again): ```bash $ docker inspect <yourContainerID> [{ "ID": "<yourContainerID>", . . . "Volumes": { "/var/webapp": "/var/lib/docker/vfs/dir/f4280c5b6207ed531efd4cc673ff620cef2a7980f747dbbcca001db61de04468" }, "VolumesRW": { "/var/webapp": true }, }] ``` * We can see that our volume is present on the file system of the Docker host. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Sharing a single file The same `-v` flag can be used to share a single file (instead of a directory). One of the most interesting examples is to share the Docker control socket. ```bash $ docker run -it -v /var/run/docker.sock:/var/run/docker.sock docker sh ``` From that container, you can now run `docker` commands communicating with the Docker Engine running on the host. Try `docker ps`! .warning[Since that container has access to the Docker socket, it has root-like access to the host.] .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Volume plugins You can install plugins to manage volumes backed by particular storage systems, or providing extra features. For instance: * [dvol](https://github.com/ClusterHQ/dvol) - allows to commit/branch/rollback volumes; * [Flocker](https://clusterhq.com/flocker/introduction/), [REX-Ray](https://github.com/emccode/rexray) - create and manage volumes backed by an enterprise storage system (e.g. SAN or NAS), or by cloud block stores (e.g. EBS); * [Blockbridge](http://www.blockbridge.com/), [Portworx](http://portworx.com/) - provide distributed block store for containers; * and much more! .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- ## Section summary We've learned how to: * Create and manage volumes. * Share volumes across containers. * Share a host directory with one or many containers. .debug[[intro/Working_With_Volumes.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Working_With_Volumes.md)] --- class: pic .interstitial[] --- name: toc-compose-for-development-stacks class: title Compose for development stacks .nav[ [Previous section](#toc-working-with-volumes) | [Back to table of contents](#toc-chapter-4) | [Next section](#toc-advanced-dockerfiles) ] .debug[(automatically generated title slide)] --- # Compose for development stacks Dockerfiles are great to build container images. But what if we work with a complex stack made of multiple containers? Eventually, we will want to write some custom scripts and automation to build, run, and connect our containers together. There is a better way: using Docker Compose. In this section, you will use Compose to bootstrap a development environment. .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## What is Docker Compose? Docker Compose (formerly known as `fig`) is an external tool. Unlike the Docker Engine, it is written in Python. It's open source as well. The general idea of Compose is to enable a very simple, powerful onboarding workflow: 1. Checkout your code. 2. Run `docker-compose up`. 3. Your app is up and running! .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Compose overview This is how you work with Compose: * You describe a set (or stack) of containers in a YAML file called `docker-compose.yml`. * You run `docker-compose up`. * Compose automatically pulls images, builds containers, and starts them. * Compose can set up links, volumes, and other Docker options for you. * Compose can run the containers in the background, or in the foreground. * When containers are running in the foreground, their aggregated output is shown. Before diving in, let's see a small example of Compose in action. .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Compose in action  .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Checking if Compose is installed If you are using the official training virtual machines, Compose has been pre-installed. You can always check that it is installed by running: ```bash $ docker-compose --version ``` .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Launching Our First Stack with Compose First step: clone the source code for the app we will be working on. ```bash $ cd $ git clone git://github.com/jpetazzo/trainingwheels ... $ cd trainingwheels ``` Second step: start your app. ```bash $ docker-compose up ``` Watch Compose build and run your app with the correct parameters, including linking the relevant containers together. .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Launching Our First Stack with Compose Verify that the app is running at `http://<yourHostIP>:8000`.  .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Stopping the app When you hit `^C`, Compose tries to gracefully terminate all of the containers. After ten seconds (or if you press `^C` again) it will forcibly kill them. .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## The `docker-compose.yml` file Here is the file used in the demo: .small[ ```yaml version: "2" services: www: build: www ports: - 8000:5000 user: nobody environment: DEBUG: 1 command: python counter.py volumes: - ./www:/src redis: image: redis ``` ] .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Compose file versions Version 1 directly has the various containers (`www`, `redis`...) at the top level of the file. Version 2 has multiple sections: * `version` is mandatory and should be `"2"`. * `services` is mandatory and corresponds to the content of the version 1 format. * `networks` is optional and indicates to which networks containers should be connected. <br/>(By default, containers will be connected on a private, per-app network.) * `volumes` is optional and can define volumes to be used and/or shared by the containers. Version 3 adds support for deployment options (scaling, rolling updates, etc.) .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Containers in `docker-compose.yml` Each service in the YAML file must contain either `build`, or `image`. * `build` indicates a path containing a Dockerfile. * `image` indicates an image name (local, or on a registry). * If both are specified, an image will be built from the `build` directory and named `image`. The other parameters are optional. They encode the parameters that you would typically add to `docker run`. Sometimes they have several minor improvements. .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Container parameters * `command` indicates what to run (like `CMD` in a Dockerfile). * `ports` translates to one (or multiple) `-p` options to map ports. <br/>You can specify local ports (i.e. `x:y` to expose public port `x`). * `volumes` translates to one (or multiple) `-v` options. <br/>You can use relative paths here. For the full list, check: https://docs.docker.com/compose/compose-file/ .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Compose commands We already saw `docker-compose up`, but another one is `docker-compose build`. It will execute `docker build` for all containers mentioning a `build` path. It can also be invoked automatically when starting the application: ```bash docker-compose up --build ``` Another common option is to start containers in the background: ```bash docker-compose up -d ``` .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Check container status It can be tedious to check the status of your containers with `docker ps`, especially when running multiple apps at the same time. Compose makes it easier; with `docker-compose ps` you will see only the status of the containers of the current stack: ```bash $ docker-compose ps Name Command State Ports ---------------------------------------------------------------------------- trainingwheels_redis_1 /entrypoint.sh red Up 6379/tcp trainingwheels_www_1 python counter.py Up 0.0.0.0:8000->5000/tcp ``` .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Cleaning up (1) If you have started your application in the background with Compose and want to stop it easily, you can use the `kill` command: ```bash $ docker-compose kill ``` Likewise, `docker-compose rm` will let you remove containers (after confirmation): ```bash $ docker-compose rm Going to remove trainingwheels_redis_1, trainingwheels_www_1 Are you sure? [yN] y Removing trainingwheels_redis_1... Removing trainingwheels_www_1... ``` .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Cleaning up (2) Alternatively, `docker-compose down` will stop and remove containers. It will also remove other resources, like networks that were created for the application. ```bash $ docker-compose down Stopping trainingwheels_www_1 ... done Stopping trainingwheels_redis_1 ... done Removing trainingwheels_www_1 ... done Removing trainingwheels_redis_1 ... done ``` .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Special handling of volumes Compose is smart. If your container uses volumes, when you restart your application, Compose will create a new container, but carefully re-use the volumes it was using previously. This makes it easy to upgrade a stateful service, by pulling its new image and just restarting your stack with Compose. .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Compose project name * When you run a Compose command, Compose infers the "project name" of your app. * By default, the "project name" is the name of the current directory. * For instance, if you are in `/home/zelda/src/ocarina`, the project name is `ocarina`. * All resources created by Compose are tagged with this project name. * The project name also appears as a prefix of the names of the resources. E.g. in the previous example, service `www` will create a container `ocarina_www_1`. * The project name can be overridden with `docker-compose -p`. .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- ## Running two copies of the same app If you want to run two copies of the same app simultaneously, all you have to do is to make sure that each copy has a different project name. You can: * copy your code in a directory with a different name * start each copy with `docker-compose -p myprojname up` Each copy will run in a different network, totally isolated from the other. This is ideal to debug regressions, do side-by-side comparisons, etc. .debug[[intro/Compose_For_Dev_Stacks.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Compose_For_Dev_Stacks.md)] --- class: pic .interstitial[] --- name: toc-advanced-dockerfiles class: title Advanced Dockerfiles .nav[ [Previous section](#toc-compose-for-development-stacks) | [Back to table of contents](#toc-chapter-4) | [Next section](#toc-links-and-resources) ] .debug[(automatically generated title slide)] --- # Advanced Dockerfiles  .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## Objectives We have seen simple Dockerfiles to illustrate how Docker build container images. In this section, we will see more Dockerfile commands. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## `Dockerfile` usage summary * `Dockerfile` instructions are executed in order. * Each instruction creates a new layer in the image. * Docker maintains a cache with the layers of previous builds. * When there are no changes in the instructions and files making a layer, the builder re-uses the cached layer, without executing the instruction for that layer. * The `FROM` instruction MUST be the first non-comment instruction. * Lines starting with `#` are treated as comments. * Some instructions (like `CMD` or `ENTRYPOINT`) update a piece of metadata. (As a result, each call to these instructions makes the previous one useless.) .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `MAINTAINER` instruction The `MAINTAINER` instruction tells you who wrote the `Dockerfile`. ```dockerfile MAINTAINER Docker Education Team <education@docker.com> ``` It's optional but recommended. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `RUN` instruction The `RUN` instruction can be specified in two ways. With shell wrapping, which runs the specified command inside a shell, with `/bin/sh -c`: ```dockerfile RUN apt-get update ``` Or using the `exec` method, which avoids shell string expansion, and allows execution in images that don't have `/bin/sh`: ```dockerfile RUN [ "apt-get", "update" ] ``` .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## More about the `RUN` instruction `RUN` will do the following: * Execute a command. * Record changes made to the filesystem. * Work great to install libraries, packages, and various files. `RUN` will NOT do the following: * Record state of *processes*. * Automatically start daemons. If you want to start something automatically when the container runs, you should use `CMD` and/or `ENTRYPOINT`. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## Collapsing layers It is possible to execute multiple commands in a single step: ```dockerfile RUN apt-get update && apt-get install -y wget && apt-get clean ``` It is also possible to break a command onto multiple lines: It is possible to execute multiple commands in a single step: ```dockerfile RUN apt-get update \ && apt-get install -y wget \ && apt-get clean ``` .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `EXPOSE` instruction The `EXPOSE` instruction tells Docker what ports are to be published in this image. ```dockerfile EXPOSE 8080 EXPOSE 80 443 EXPOSE 53/tcp 53/udp ``` * All ports are private by default. * Declaring a port with `EXPOSE` is not enough to make it public. * The `Dockerfile` doesn't control on which port a service gets exposed. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## Exposing ports * When you `docker run -p <port> ...`, that port becomes public. (Even if it was not declared with `EXPOSE`.) * When you `docker run -P ...` (without port number), all ports declared with `EXPOSE` become public. A *public port* is reachable from other containers and from outside the host. A *private port* is not reachable from outside. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `COPY` instruction The `COPY` instruction adds files and content from your host into the image. ```dockerfile COPY . /src ``` This will add the contents of the *build context* (the directory passed as an argument to `docker build`) to the directory `/src` in the container. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## Build context isolation Note: you can only reference files and directories *inside* the build context. Absolute paths are taken as being anchored to the build context, so the two following lines are equivalent: ```dockerfile COPY . /src COPY / /src ``` Attempts to use `..` to get out of the build context will be detected and blocked with Docker, and the build will fail. Otherwise, a `Dockerfile` could succeed on host A, but fail on host B. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## `ADD` `ADD` works almost like `COPY`, but has a few extra features. `ADD` can get remote files: ```dockerfile ADD http://www.example.com/webapp.jar /opt/ ``` This would download the `webapp.jar` file and place it in the `/opt` directory. `ADD` will automatically unpack zip files and tar archives: ```dockerfile ADD ./assets.zip /var/www/htdocs/assets/ ``` This would unpack `assets.zip` into `/var/www/htdocs/assets`. *However,* `ADD` will not automatically unpack remote archives. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## `ADD`, `COPY`, and the build cache * Before creating a new layer, Docker checks its build cache. * For most Dockerfile instructions, Docker only looks at the `Dockerfile` content to do the cache lookup. * For `ADD` and `COPY` instructions, Docker also checks if the files to be added to the container have been changed. * `ADD` always needs to download the remote file before it can check if it has been changed. (It cannot use, e.g., ETags or If-Modified-Since headers.) .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## `VOLUME` The `VOLUME` instruction tells Docker that a specific directory should be a *volume*. ```dockerfile VOLUME /var/lib/mysql ``` Filesystem access in volumes bypasses the copy-on-write layer, offering native performance to I/O done in those directories. Volumes can be attached to multiple containers, allowing to "port" data over from a container to another, e.g. to upgrade a database to a newer version. It is possible to start a container in "read-only" mode. The container filesystem will be made read-only, but volumes can still have read/write access if necessary. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `WORKDIR` instruction The `WORKDIR` instruction sets the working directory for subsequent instructions. It also affects `CMD` and `ENTRYPOINT`, since it sets the working directory used when starting the container. ```dockerfile WORKDIR /src ``` You can specify `WORKDIR` again to change the working directory for further operations. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `ENV` instruction The `ENV` instruction specifies environment variables that should be set in any container launched from the image. ```dockerfile ENV WEBAPP_PORT 8080 ``` This will result in an environment variable being created in any containers created from this image of ```bash WEBAPP_PORT=8080 ``` You can also specify environment variables when you use `docker run`. ```bash $ docker run -e WEBAPP_PORT=8000 -e WEBAPP_HOST=www.example.com ... ``` .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `USER` instruction The `USER` instruction sets the user name or UID to use when running the image. It can be used multiple times to change back to root or to another user. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `CMD` instruction The `CMD` instruction is a default command run when a container is launched from the image. ```dockerfile CMD [ "nginx", "-g", "daemon off;" ] ``` Means we don't need to specify `nginx -g "daemon off;"` when running the container. Instead of: ```bash $ docker run <dockerhubUsername>/web_image nginx -g "daemon off;" ``` We can just do: ```bash $ docker run <dockerhubUsername>/web_image ``` .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## More about the `CMD` instruction Just like `RUN`, the `CMD` instruction comes in two forms. The first executes in a shell: ```dockerfile CMD nginx -g "daemon off;" ``` The second executes directly, without shell processing: ```dockerfile CMD [ "nginx", "-g", "daemon off;" ] ``` .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- class: extra-details ## Overriding the `CMD` instruction The `CMD` can be overridden when you run a container. ```bash $ docker run -it <dockerhubUsername>/web_image bash ``` Will run `bash` instead of `nginx -g "daemon off;"`. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## The `ENTRYPOINT` instruction The `ENTRYPOINT` instruction is like the `CMD` instruction, but arguments given on the command line are *appended* to the entry point. Note: you have to use the "exec" syntax (`[ "..." ]`). ```dockerfile ENTRYPOINT [ "/bin/ls" ] ``` If we were to run: ```bash $ docker run training/ls -l ``` Instead of trying to run `-l`, the container will run `/bin/ls -l`. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- class: extra-details ## Overriding the `ENTRYPOINT` instruction The entry point can be overriden as well. ```bash $ docker run -it training/ls bin dev home lib64 mnt proc run srv tmp var boot etc lib media opt root sbin sys usr $ docker run -it --entrypoint bash training/ls root@d902fb7b1fc7:/# ``` .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## How `CMD` and `ENTRYPOINT` interact The `CMD` and `ENTRYPOINT` instructions work best when used together. ```dockerfile ENTRYPOINT [ "nginx" ] CMD [ "-g", "daemon off;" ] ``` The `ENTRYPOINT` specifies the command to be run and the `CMD` specifies its options. On the command line we can then potentially override the options when needed. ```bash $ docker run -d <dockerhubUsername>/web_image -t ``` This will override the options `CMD` provided with new flags. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- ## Advanced Dockerfile instructions * `ONBUILD` lets you stash instructions that will be executed when this image is used as a base for another one. * `LABEL` adds arbitrary metadata to the image. * `ARG` defines build-time variables (optional or mandatory). * `STOPSIGNAL` sets the signal for `docker stop` (`TERM` by default). * `HEALTHCHECK` defines a command assessing the status of the container. * `SHELL` sets the default program to use for string-syntax RUN, CMD, etc. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- class: extra-details ## The `ONBUILD` instruction The `ONBUILD` instruction is a trigger. It sets instructions that will be executed when another image is built from the image being build. This is useful for building images which will be used as a base to build other images. ```dockerfile ONBUILD COPY . /src ``` * You can't chain `ONBUILD` instructions with `ONBUILD`. * `ONBUILD` can't be used to trigger `FROM` and `MAINTAINER` instructions. .debug[[intro/Advanced_Dockerfiles.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/intro/Advanced_Dockerfiles.md)] --- class: title, self-paced Thank you! .debug[[common/thankyou.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/thankyou.md)] --- class: title, in-person That's all folks! <br/> Questions?  .debug[[common/thankyou.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/thankyou.md)] --- class: pic .interstitial[] --- name: toc-links-and-resources class: title Links and resources .nav[ [Previous section](#toc-advanced-dockerfiles) | [Back to table of contents](#toc-chapter-4) | [Next section](#toc-) ] .debug[(automatically generated title slide)] --- # Links and resources - [Docker Community Slack](https://community.docker.com/registrations/groups/4316) - [Docker Community Forums](https://forums.docker.com/) - [Docker Hub](https://hub.docker.com) - [Docker Blog](http://blog.docker.com/) - [Docker documentation](http://docs.docker.com/) - [Docker on StackOverflow](https://stackoverflow.com/questions/tagged/docker) - [Docker on Twitter](http://twitter.com/docker) - [Play With Docker Hands-On Labs](http://training.play-with-docker.com/) .footnote[These slides (and future updates) are on → http://container.training/] .debug[[common/thankyou.md](https://github.com/jpetazzo/container.training/tree/qconsf2017intro/slides/common/thankyou.md)]