Docker Internals Visualized I: Docker Build process
What happens when you enter the docker buld command. Let's learn visually.
Welcome to the 30 new followers who have subscribed since last week.
We all know what Docker is and why it is used. As part of my Docker Internal Visualized series, we will learn how Docker works internally.
In the first part, we will explore how the Docker build process works internally, from image creation to layer creation.
1. Docker Build Command
docker build -t node-express .This command creates a Docker image for a project using a Docker file.
Let’s break down this command.
1. docker build
Here, docker build informs the Docker client that you want to build an image.
2. -t 🏷️
-t refers to adding a tag to the image after it is created in the name:tag format. If no tag is specified, it will add the latest tag.
3. node-express
This refers to the name to be assigned to the image being built. It helps out later in identifying images by readable names.
4. “.”
The dot added at the end specifies the build context. When “.” is added, it refers to the current directory used as the build context.
.
├── index.ts
├── src/
├── Dockerfile
├── package.json
└── package-lock.jsonBuild context
The build context is just a set of files at a location that Docker will use to build. These files can be the project files against which the Docker daemon will create a build.
2. Docker client creating TAR file
After the docker build command gets entered, the Docker client comes into play.
It copies the build context files excluding those mentioned in .dockerignore and converts those into a TAR file.
The build time depends upon the size of the files inside the directory and it is essential to add unnecessary files in .dockerignore.
3. Docker Daemon (Dockerd) process TAR file
Docker client is a user-friendly interface to communicate with Docker Daemon via REST-API call.
When the docker build command is executed, under the hood, the Docker client sends the TAR file to Dockerd via an internal REST API call.
The Dockerd then unpacks, parses and loads the TAR file for executing each instruction one-by-one as per the Dockerfile and building the required Docker image.
4. Docker Filesystem and hash
Before moving further, it is important to understand how a docker image is internally built up.
Docker images are layers of files. Each layer represents a set of file changes. Each instruction in the Dockerfile such as FROM, WORKDIR, etc creates a layer after modifying the filesystem.
Each layer is created once and can be shared across multiple images.
Docker creates these layers because they are:
Immutable once created.
Reusable across different builds and efficient storage management.
Layers Stacking
FROM instruction creates a base layer and all other layers are built on top of it.
The subsequent instructions create and stack layers in the same order as defined in the Dockerfile.
Docker File Layer Hashing
When a new layer is created, Docker attaches a checksum SHA-256 hash. The hash is generated using metadata and file content internally by Docker itself.
It helps
uniquely identify layers,
securing from tempering because if some file changes the Docker will create a new hash and that image will not exploited.
verification
caching and improving latency and efficiency
We will discuss these concepts with examples ahead.
Note: In the animations, the base layer is on top and the following layers are below it, which is a visual representation mistake and the correct one will be updated in the next part of the series.
5. Dockerfile execution in Dockerd
FROM node:18-alpine
As discussed above, Docker Engine will first check the layered file on the local machine and if it doesn’t match any layer, it will
pull an image from DockerHub
create a new layer
attach a SHA-256 hash to it.
Loading Build Context
The Docker will load the Build Context inside the execution context because the image will be created against these files.
WORKDIR /app
The WORKDIR command specifies the working directory to execute its subsequent instructions such as COPY, RUN, CMD, ADD, ENTRYPOINT, etc.
If the directory doesn’t exist, Docker will create this directory and all subsequent relative paths will be on top of this directory.
If this path is not specified, the Docker sets it at “/” by default.
The layer created at this step is very minimal and is empty and mainly modifies the metadata of the image. However, it is important to understand that the layer while not adding up to the storage, is effective for leveraging cached layers
COPY . .
This command will copy files from the current directory which are usually in the current directory to the previously defined WORKDIR, /app in this case.
With this command, all files in the build context are copied to the /app or whatever directory specified.
yarn install
It will simply run the command “yarn install” inside the /app directory where the project files are copied. It will also create a succeeding layer when the yarn install is successful.
CMD & EXPOSE
CMD instruction will set the command, you want to run when the container of this image will run. Similarly, EXPOSE exposes the port to connect with the container.
Here, the CMD and EXPOSE are empty layers similar to WORKDIR.
At this step our Docker image is ready.
6. Docker Image Layers
Let’s discuss Docker image layers.
To analyze Docker layers, you can run this command:
$ sudo docker history image-idIt will enlist all filesystem layers of the image.
As evident each layer represents a corresponding command in Dockerfile. Some add up to storage and others modify metadata.
Docker instructions such as WORKDIR, EXPOSE, and CMD, don’t add up to storage and modify the underlying image metadata.
All of the layers are stacked according to the instructions.
If a Dockerfile with the same instructions is used, the image will be created from cached layers including the base image pulled from Dockerhub.
In the next part of the series, we will explore Docker Caching and Invalidation in-depth with multiple use cases such as copy-on-write.
If you liked this post, please like, subscribe and restack.
Cheers.