Zhang Haowei

Zhang Haowei

Welcome here!

Introduction to Docker

8 minute read

Published:

A step by step introduction to git. Also, see reference1

Outline

  1. What is Docker (DevOps=Devlops+Operations)
    • Image: Like a templete or class, can be used to create (multiple) container. E.g., tomcat image ==> run ==> tomcat01 container (provide server).
    • Container: Created by image (object). Including basic commands as run, stop, delete, etc. Like a simplified linux system.
    • Repository: The place to deposit images, including public and private repositories.
  2. Install Docker @ Ubuntu 22.04 LTS
    • Prerequisites
      • $ sudo apt-get remove docker docker-engine docker.io containerd runc
    • Install Docker Engine
      • Update the apt package index and install packages to allow apt to use a repository over HTTPS:
        • $ sudo apt-get update
        • $ sudo apt-get install ca-certificates curl gnupg lsb-release
      • Add Docker’s official GPG key:
        • $ sudo mkdir -p /etc/apt/keyrings
        • Official: $ curl -fsSL https://download.docker.com/linux/ubuntu/gpgsudo gpg –dearmor -o /etc/apt/keyrings/docker.gpg
        • Use aliyun (optional): $ curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpgsudo gpg –dearmor -o /etc/apt/keyrings/docker.gpg
      • Use the following command to set up the repository:
        • Official: $ echo
          “deb [arch=$(dpkg –print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu
          $(lsb_release -cs) stable” | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
        • Use aliyun (optional): $ echo
          “deb [arch=$(dpkg –print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://mirrors.aliyun.com/docker-ce/linux/ubuntu
          $(lsb_release -cs) stable” | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
      • Install Docker Engine
        • $ sudo apt-get update
        • $ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
      • Change the mirror of Docker Hub to aliyun (optional):
        • $ sudo mkdir -p /etc/docker
        • $ sudo tee /etc/docker/daemon.json «-‘EOF’
          {
          “registry-mirrors”: [“https://zjh3aomb.mirror.aliyuncs.com”]
          }
          EOF
        • $ sudo systemctl daemon-reload
        • $ sudo systemctl restart docker
      • Test and verify
        • $ sudo docker version
        • $ sudo docker run hello-world
      • Manage Docker as a non-root user
    • Change the default storage location of Docker images
      • Check the present location
        • $ sudo docker info | grep Dir
      • Shutdown the Docker service
        • $ sudo systemctl stop docker
        • $ sudo systemctl status docker
      • Move the data to the new location, e.g., /lustre/haowei/Docker/lib/docker
        • $ sudo mv /var/lib/docker /lustre/haowei/Docker/lib/
      • Modify the docker.service ($ sudo vim /lib/systemd/system/docker.service) startup configuration file using the –graph parameter to specify the storage location
        • Change
        • ExecStart=/usr/bin/dockerd -H fd:// –containerd=/run/containerd/containerd.sock
        • into
        • ExecStart=/usr/bin/dockerd -H fd:// –containerd=/run/containerd/containerd.sock –graph /lustre/haowei/Docker/lib/docker
      • Reload the configuration file
        • $ sudo systemctl daemon-reload
      • Restart Docker service & check
        • $ sudo systemctl start docker
        • $ sudo systemctl enable docker
        • $ sudo systemctl status docker
        • $ sudo docker run hello-world
    • [Uninstall Docker Engine] (https://docs.docker.com/engine/install/ubuntu/) and also [this reference] (https://askubuntu.com/questions/935569/how-to-completely-uninstall-docker)
      • $ sudo apt-get purge docker-ce docker-ce-cli containerd.io docker-compose-plugin
      • $ sudo rm -rf /var/lib/docker
      • $ sudo rm -rf /var/lib/containerd
  3. Docker commands
    • $ sudo docker xxx –help
    • $ sudo docker version
    • $ sudo docker info: system information of docker, including the number of mirrors and containers
    • Image commands
    • Container commands
      • $ sudo docker run: Run a command in a new container
        • –name=”aName”: the name of the running container
        • -d: run on background, same as the nohup
        • -it: run in interactive mode and enter the container to view the contents
        • -c: follow a bash command, e.g. $ sudo docker run -d centos /bin/bash -c “while true;do echo kuangshen;sleep 1;done”
        • -p: specify and expose the port 8080:8080
          • -p ip : hostPort : containerPort (most used)
          • -p hostPort : containerPort (most used)
          • -p containerPort
          • containerPort
        • -P: specify the port randomly
        • –rm: delete the container when exit
        • $ sudo docker ps: see the running container
          • $ sudo docker ps -a: see the run container history
        • E.g., run and enter intel/oneapi-basekit: $ sudo docker run -it intel/oneapi-basekit /bin/bash
      • Exit container
      • $ sudo docker rm containerID: Remove container (such as object)
      • Start/stop containers
        • $ sudo docker start containerID
        • $ sudo docker restart containerID
        • $ sudo docker stop containerID
        • $ sudo docker kill containerID
    • Other useful commands
    • Docker commands relation
  4. Visualization
    • portainer (Docker graphical interface management tool)
      • $ sudo docker run -d -p 8088:9000 –restart=always -v /var/run/docker.sock:/var/run/docker.sock –privileged=true portainer/portainer
      • visit http://10.72.197.103:8088/, set password and select the local choice
    • Rancher (CI/CD)
      • ……
  5. Docker commit
    • To save the modified container as a new docker image, use commit as:
      • $ sudo docker commit -m=”your message” -a=”author” containerID dockerName:versionTag
  6. Docker - Data Volume
    • Synchronize the data between container and host, achieved by mount the directory of the container into the host. The directory can also be shared between different containers.
      • Method 1: $ sudo docker run -it -v hostDirectory:containerDirectory
        • E.g. $ sudo docker run -it -v /home/haowei/test:/home centos
        • Direct mount: -v hostDirectory:containerDirectory
        • Implicit mount: -v /containerDirectory
        • Explicit mount: -v volumeName:/containerDirectory
        • Share the mount between host and several containers with <–volumes-from>:
          • $ sudo docker run -it –name centos01 -v /home/haowei/centosHome:/home centos
          • $ sudo docker run -it –name centos02 –volumes-from centos01 centos
        • Specific permissions:
          • Read only: -v hostDirectory:containerDirectory:ro
          • Read and write: -v hostDirectory:containerDirectory:rw
        • Check the mount information:
          • $ sudo docker volume ls
          • $ sudo docker volume inspect volumeName
  7. DockerFile
    • Create your own mirrors, steps:
      • Write a dockfile file
      • docker build: create the mirror
      • docker run: execute your mirror as a container
      • docker push: publish your mirror (DockerHub or aliyun)
    • DockerFile commands:
      • FROM: the base mirror
      • MAINTAINER: name+email
      • RUN: the commands to be executed when build
      • ADD: add content/file
      • WORKDIR: the workdir of mirror/container
      • VOLUME: the mounted directory
      • EXPOSE: reserve the port settings
      • CMD: the commands to be executed when container is started, only the last one will be executed (replace)
      • ENTRYPOINT: the commands to be executed when container is started, we can add more commands after run the container (append)
      • ONBUILD: when build a heritable DockFile, the ONBUILD command will be executed.
      • COPY: copy files to the mirror
      • ENV: set environment when build

    • Example01: create a new ubuntu

        FROM ubuntu
  MAINTAINER haowei<changhw@zju.edu.cn>
  ENV MYPATH /usr/local
  WORKDIR $MYPATH
        
  RUN touch aFile
  #RUN sed -i 's/archive.ubuntu.com/mirrors.ustc.edu.cn/g' /etc/apt/sources.list            
  #RUN apt update
  #RUN apt -y install vim
        
  EXPOSE 80

  CMD echo $MYPATH
  CMD /bin/bash
      • Build with: $ sudo docker build -f mydockerfile-ubuntu -t myubuntu:latest .
        • write a Dockerfile, ‘-f’ flag will be ignored if the name of Dockerfile is ‘Dockerfile’
      • See the Docker image history with: $ sudo docker history imageID
    • Example02: create a new ubuntu with oneAPI (something went wrong)

      • $ sudo docker build -t ubuntuonepai:latest .

          From ubuntu
  MAINTAINER haowei<changhw@zju.edu.cn>

  COPY readme.txt /usr/local/readme.txt
  COPY l_BaseKit_p_2022.2.0.262_offline.sh /root/

  ENV MYPATH /root/
  WORKDIR $MYPATH

  RUN sh l_BaseKit_p_2022.2.0.262_offline.sh

  EXPOSE 8080

    • Example03: create a docker with oneapi samples

        # run the development container and name it mybuild
  FROM intel/oneapi-basekit:devel-ubuntu20.04 as mybuild

  # get oneAPI sample code
  RUN git clone https://github.com/oneapi-src/oneAPI-samples

  # build the Nbody sample to root directory
  RUN cmake /oneAPI-samples/DirectProgramming/DPC++/N-BodyMethods/Nbody
  RUN make

  # use oneapi-runtime container as my production container
  FROM intel/oneapi-runtime:latest

  # copy file from mybuild to the production container
  COPY - from=mybuild src/nbody /
  CMD ["/nbody"]
  8. Publish your Docker images
    • DockerHub
      • login: $ sudo docker login -u changhw
      • push: $ sudo docker push changhw/changhwubuntu:0.0.0
        • if it is denied, change the name of image, such as
          • $ sudo docker tag 0701f0a76ec1 changhw/changhwubuntu:0.0.0
    • Aliyun
  9. Docker Network Principle
  10. Docker Compose
  11. Docker Swarm
  12. CI/CD Jenkins
  13. Else
    • Pull the Nvidia hpcsdk container: $ sudo docker run –gpus all -it –rm nvcr.io/nvidia/nvhpc:22.5-devel-cuda_multi-ubuntu20.04

You May Also Enjoy

CUDA study

2 minute read

Published:

Following CUDA 编程 （樊哲勇）.

Python study

less than 1 minute read

Published:

Just some adopt scripts or commands.