Tag Archives: docker

Loading container images into Kubernetes containerd

Do you need to run containers whose images are not available in a container registry? Here’s how…

Make sure you have the image available as a tar file. Here’s how to save one from a machine with docker:

docker save repository/image --output ./image.tar

Copy the tarfile to the target machine and run this command to load image into containerd:

sudo ctr -n k8s.io images import image.tar

To use this image in a Kubernetes deployment, make sure its marked as never pull:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: my-app
  labels:
    app: my-app

spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: my-app
          image: xxxxx/my-app
          imagePullPolicy: Never

How to install Kubernetes onto physical machines for a home lab

On each machine: Install Ubuntu Server LTS 24.04

Ensure you can SSH into it and enable password less sudo

echo "$USER ALL=(ALL:ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/$USER

This helps in running commands on each machine in parallel.

On each machine: Install kubeadm

Based on Bootstrapping clusters with kubeadm

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gnupg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | $ sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo swapoff -a

On each machine: Install containerd

Kubernetes recently deprecated usage of dockerd as the container runtime. So we’ll use containerd directly based on Anthony Nocentino’s blog: Installing and Configuring containerd as a Kubernetes Container Runtime

Configure the required kernel modules:

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter

Configure persistence across system reboots

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

Install containerd packages

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update && sudo apt-get install containerd.io

Create a containerd configuration file

sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml

Set the cgroup driver to systemd. Kuberbetes uses systemd while containerd uses something else. They must both use the same setting:

sudo sed -i 's/            SystemdCgroup = false/            SystemdCgroup = true/' /etc/containerd/config.toml
sudo systemctl restart containerd
sudo systemctl enable containerd

Only on the first (master) machine

Initialize the K8s cluster. Save this output somewhere, you’ll need the kubeadm join ... part later.

sudo kubeadm init --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.30.2
[preflight] Running pre-flight checks
...
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
W0629 19:20:06.570522   14350 checks.go:844] detected that the sandbox image "registry.k8s.io/pause:3.8" of the container runtime is inconsistent with that used by kubeadm.It is recommended to use "registry.k8s.io/pause:3.9" as the CRI sandbox image.
...
Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.178.171:6443 --token v1flk8.wy9xyikw6kosevps \
        --discovery-token-ca-cert-hash sha256:e79a8516a0990fa232b6dcde15ed951ffe46880854fe1169ceb3b909d82fff00

On each machine: Follow the recommendation of kubeadmin to update the sandbox image.

Use a text editor to replace sandbox_image = "registry.k8s.io/pause:3.8" with sandbox_image = "registry.k8s.io/pause:3.9"

restart containerd

sudo systemctl restart containerd.service

On the master node: Ensure kubectl knows what cluster you work with

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

On each other machine: join them to the cluster:

kubeadm join 192.168.178.171:6443 \ 
    --token v1flk8.wy9xyikw6kosevps \
    --discovery-token-ca-cert-hash sha256:e79a8516a0990fa232b6dcde15ed951ffe46880854fe1169ceb3b909d82fff00

On the master node: Configure the POD network

wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
kubectl apply -f kube-flannel.yml

Installation finished, check status:

kubectl get nodes

should give output like:

NAME        STATUS   ROLES           AGE     VERSION
optiplex1   Ready    control-plane   2d23h   v1.30.2
optiplex2   Ready    <none>          2d23h   v1.30.2
optiplex3   Ready    <none>          2d23h   v1.30.2

Zalenium a stable and scalable Selenium grid

I just want to give well deserved thumbs up to Zalando’s Zalenium Their own description says it best:

Allows anyone to have a disposable, flexible, container based Selenium Grid infrastructure featuring video recording, live preview, basic auth & online/offline dashboards

Getting up and-running really is only one docker pull and docker run command away.

Accessing gpio pins inside a docker container on a raspberry pi

If your container needs access to the GPIO pins, then it must have access to the /dev/gpiomem device. From the command line you can do that like this:

$ docker run --device=/dev/gpiomem:/dev/gpiomem ...rest of commandline...

Here’s how to do it with a docker-compose file:

version: "2"

services:
  app:
    devices:
      - /dev/gpiomem:/dev/gpiomem
    ports:
     ...rest of the file...

Containerising the development environment

One of the nice things about docker is that we can use all kinds of software without cluttering up our local machine. I really like the ability to have the development environment running in a container. Here is an example where we:

  • Get a Node.js development environment with all required tools and packages
  • Allow remote debugging of the app in the container
  • See code changes immediately reflected inside the container

The dockerfile below gives us a container with all required tools and packages for a Node.js app. In this example we assume the ‘.’ directory contains the files needed to run the app.

FROM node:9

WORKDIR /code

RUN npm install -g nodemon

COPY package.json /code/package.json
RUN npm install && npm ls
RUN mv /code/node_modules /node_modules
COPY . /code

CMD ["npm", "start"]

That’s nice, but how does this provide remote debugging? and how do code changes propagate to a running container?

Two very normal aspects of docker achieve this. Firstly docker-compose.yml overrules the CMD ["npm", "start"] statement to start nodemon with the --inspect=0.0.0.5858 flag. That starts the app with the debugger listening on all of the machines IP addresses. We expose port 5858 to allow remote debuggers to connect to the app in the container.

Secondly, the compose file contains a volume mapping that overrules the /code folder in the container and points it to the directory on the local machine where you edit the code. Combined with the --watch flag nodemon sees any changes you make to the code and restarts the app in the container with the latest code changes.

Note: If you are running docker on Windows of the code is stored on some network share, then you must use the --legacy-watch flag instead of --watch

The docker-compose.yml file:

version: "2"

services:
  app:
    build: .
    command: nodemon --inspect=0.0.0.0:5858 --watch
    volumes:
      - ./:/code
    ports:
      - "5858:5858"

Here’s a launch.json for Visual Studio Code to attach to the container.

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Attach",
            "type": "node",
            "request": "attach",
            "port": 5858,
            "address": "localhost",
            "restart": true,
            "sourceMaps": false,
            "outDir": null,
            "localRoot": "${workspaceRoot}",
            "remoteRoot": "/code"
        }
    ]
}

Docker on Raspbian: cgroup not supported on this system

Are you running Docker on Raspbian and getting the error:

cgroups: memory cgroup not supported on this system

Best solution is to add cgroup_memory=1 in /boot/cmdline.txt and reboot.

sudo echo "cgroup_memory=1" >> /boot/cmdline.txt

PLease note, for future releases of Raspbian you will need the following instead:

sudo echo "cgroup_enable=memory" >> /boot/cmdline.txt

Alternatively, you can downgrade to an earlier docker version:

sudo apt-get install -y docker-ce=17.09.0~ce-0~raspbian --allow-downgrades