DockerInDeep

Note that this repo is under improvement, so take a pomegranate and have a nice day.
You can contribute; check the steps.

Table of content

1. Why did the concept of virtualization appear in 1960?.
2. Virtualization.
3. Brief about Base Image and the Differencing Disk in virtual machines.
4. Containers.
5. How do containers depend on the kernel? Or what technology in the kernel enables this?
6. The technology that is provided to deal with this container concept.
7. Crucial concept between a virtual machine and a container.
8. Docker Installation.
9. Docker Engine.
10. Docker client and Docker daemon communication.

Why did the concept of virtualization appear in 1960?

Virtualization

There is a lot of talk about virtualization, but here we will talk about hardware virtualization.

• Hardware Virtualization: This involves creating virtual machines (VMs) that emulate a complete hardware environment, including:

  • CPU.
  • memory.
  • storage.
  • network components.

• We can do this by adding another layer, which is called a hypervisor.

  • Hypervisor: software or firmware that creates and manages VMs. It allows multiple operating systems to share a single hardware host.
  • Types of Hypervisors:
    • Type 1 (Bare Metal): runs directly on the host's hardware to control the hardware and to manage guest operating systems.
      • VMware ESXi.
      • Microsoft Hyper-V.
      • KVM (Kernel-based Virtual Machine).
      • Xen.
    • Type 2 (Hosted): runs on a standard operating system, just like other computer programs. This type is usually used for testing and development.
      • Oracle VirtualBox.
      • VMware Workstation.

• Check out the figure for more explanation.

• Safaret News Magazine will answer how hardware virtualization solved the issues that we faced before, and the challenges it seeks to solve.

  

brief about Base Image and the Differencing Disk in virtual machines

Before we talk about containers, let's review an important concept from the virtual machine, which is the Base Image and the Differencing Disk.

• Base Disk: is a read-only disk image that contains the core operating system, applications, and system configurations. It serves as the template for creating multiple VMs.
• Differencing Disk:
  • Differencing disks are associated with specific virtual machines (VMs) and track the changes made to the base image during the VM's operation.
  • Each VM has its own differencing disk, allowing customization and modifications without altering the original base image.
  • Multiple VMs can use the same base image, and each VM has its own differencing disk to capture its unique modifications.
• Size Calculation: by assuming that
  • Base Image Size: 20GB
    • VM1 Differencing Disk: 5GB (assuming 5GB of changes specific to VM1)
      • Total disk usage for VM1: 20GB (base image) + 5GB (differencing disk) = 25GB
    • VM2 Differencing Disk: 10GB (assuming 10GB of changes specific to VM2)
      • Total disk usage for VM2: 20GB (base image) + 10GB (differencing disk) = 30GB
    • VM3 Differencing Disk: 7GB (assuming 7GB of changes specific to VM3)
      • Total disk usage for VM3: 20GB (base image) + 7GB (differencing disk) = 27GB

Containers

• Since any operating system has a kernel, if there's a way to use the kernel of the main machine, whether it's a physical machine or a virtual machine, and remove all the operating systems from each VM, and if we need any OS specification, we can add it as a layer on top of the host operating system kernal which will be good. that actually the container makes this.

• The same concept of the Base Image and the Differencing Disk in virtual machines is here in the containers.

How do containers depend on the kernel? Or what technology in the kernel enables this?

• First of all, let's learn about two important Linux kernel features: namespaces and cgroups.

  • Namespaces and cgroups are two important Linux kernel features that provide mechanisms for isolating and controlling processes and resources.

  • namespaces are used to provide process isolation and create lightweight containers, while cgroups are used to control resource allocation and limits for those containers. This combination allows you to run multiple isolated environments on a single host machine, each with its own set of processes and resource constraints.

    
• In the figure below, let say that the container starts with proc_1, which is RUN CONTAINER, then opens proc_2, which is OS specification, and each process will open the next one. Linux Kernal considers all of these processes as a single process. and using the namespaces technique to isolate all these processes from all operating system.

Tip

Look here for more explanation on namespaces and cgroups: cgroups&Namespaces;

The technology that is provided to deal with this container concept.

• Docker is perhaps the most well-known container platform. So this repo will take a closer look into Docker.

Crucial concept between a virtual machine and a container.

Docker Installation.

• You can use the repo docker-install, which is virified by Docker, or see the Docker Docs.

Note

In the documentation in the repository of the installer script, they mention:
The purpose of the install script is for a convenience for quickly installing the latest Docker-CE releases on the supported linux distros. It is not recommended to depend on this script for deployment to production systems. For more thorough instructions for installing on the supported distros, see the install instructions.

• After that, check if the installation is done by running docker info.
• If you faced issue is related to privilege or permissions, it's essential to ensure that your user has the necessary privileges to interact with Docker.
  • So you have to add the user to the Docker Group:-> sudo usermod -aG docker $USER
  • After running this command, restart your system for the changes to take effect.

Docker Engine.

• runc
  • For each running container, containerd creates a process called runc.
  • runc is responsible for anything related to this container such as starting, stopping, pausing, and more.
• shim containerd + runc
  • The shim acts as a bridge between the container runtime "containerd" and the actual container process "managed by runc".
  • It stays around even after the main container tool (like runc) has finished its job.How??
    • runc s responsible for starting the container process and managing the low-level details of container execution.
    • Once the container process is up and running, runc's primary role is considered done.
    • the shim doesn't terminate immediately after runc has finished its primary tasks. Instead, it remains active in the background
  • Ensures that communication channels, such as STDIN and STDOUT streams, remain open between the container and the outside world.
  • Reports back to the container manager (e.g., containerd) to keep it informed about the status and health of the container.
• So, this model achieves a loosely coupled architecture.
  • Separating the container runtime from the Docker daemon, allowing each component to perform its specialized functions.daemonless containers
  • Since the container runtime is not coupled with the Docker daemon, you can perform maintenance tasks such as upgrades or restarts on the Docker daemon without affecting the running containers managed by the container runtime.

Docker client and Docker daemon communication.

Next topic

• How to secure the communication between the Docker client and Docker daemon?

Contributing.

1. Fork the repository.
2. Clone your forked repository to your local machine.

   git clone https://github.com/Mohamed-abdalazez/DockerInDeep.git
   cd DockerInDeep
   

3. Create a new branch for your topic.

   git checkout -b feature-branch
   

4. Set upstream remote.

   git remote add upstream https://github.com/Mohamed-abdalazez/DockerInDeep.git
   

5. Make your changes as follows:
   • Review the next topic to be added here.
   • Implement your changes.
   • If you have a figure, use excalidraw to create it.
   • Place the figure.png in the assets folder.
   • Include the figure.excalidraw in the excalidraw folder.
   • Name the figure.png and figure.excalidraw with the topic name, keeping it as short as possible.
6. Push your changes.

   git add .
   git commit -m "Added: new topic"
   git push --set-upstream origin your-branch-name
   

7. Create pull request on GitHub.