install

install in China #

On master node #

sudo su  # run as root

export INSTALL_K3S_MIRROR=cn
export INSTALL_K3S_CHANNEL=latest
export INSTALL_K3S_EXEC="--container-runtime-endpoint /run/containerd/containerd.sock \
  --disable servicelb --disable traefik --disable metrics-server"
curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | sh -

# to check token on master node
sudo cat /var/lib/rancher/k3s/server/node-token

# kubeconfig file at
mkdir -p $HOME/.kube
cp -i /etc/rancher/k3s/k3s.yaml $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# change back to non-root user
mkdir -p $HOME/.kube
sudo cp -i /etc/rancher/k3s/k3s.yaml $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

On worker node #

export INSTALL_K3S_MIRROR=cn
export INSTALL_K3S_CHANNEL=latest
export K3S_URL=https://192.168.8.45:6443
export INSTALL_K3S_EXEC="--container-runtime-endpoint /run/containerd/containerd.sock"
export K3S_TOKEN=
curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | sh -

k3s default LB #

K3S has builtin load balancer klipper. Every time a service of type load balancer is created, k3s will deploy a daemonset, listening on that given port, on every host.

Disable it by adding --disable servicelb option to server node.

k3s default ingress: Traefik #

Disable it by --disable traefik during installation, it’s added as server arguments.

install MetalLB on k3s #

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.11.0/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.11.0/manifests/metallb.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 192.168.8.240-192.168.8.250

installation on worker node #

curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn INSTALL_K3S_CHANNEL=latest K3S_URL=https://192.168.8.65:6443 K3S_TOKEN=$K3S_TOKEN sh -

containerd #

k3s 的 containerd socket 路径 /run/k3s/containerd/containerd.sock

nerdctl 默认到 /run/containerd/containerd.sock 找

server installation #

air-gap install

image file, example: (version 1.20.2+k3s1)

https://github.com/k3s-io/k3s/releases/download/v1.20.2%2Bk3s1/k3s-airgap-images-amd64.tar

sudo mkdir -p /var/lib/rancher/k3s/agent/images/
sudo cp ./k3s-airgap-images-$ARCH.tar /var/lib/rancher/k3s/agent/images/

Place the k3s binary at /usr/local/bin/k3s and ensure it is executable

https://github.com/k3s-io/k3s/releases/download/v1.20.2%2Bk3s1/k3s
sudo cp k3s /usr/local/bin/

install script

wget https://get.k3s.io -O install.sh

install server node

INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC="--write-kubeconfig-mode 666 --tls-san 192.168.100.201 --node-external-ip=192.168.100.201" ./install.sh
# notice the external IP, from [1]

on the server node, get the token at /var/lib/rancher/k3s/server/node-token

sudo cat /var/lib/rancher/k3s/server/node-token

install on worker node

INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://192.168.100.201:6443 K3S_TOKEN=$TOKEN ./install.sh

server node #

rm -rf $HOME/.kube

mkdir -p $HOME/.kube
sudo cp -i /etc/rancher/k3s/k3s.yaml $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

https://rancher.com/docs/k3s/latest/en/installation/private-registry/

k3s 默认使用containerd做cri，启动时候会去找 /etc/rancher/k3s/registries.yaml，根据mirrors字段生成containerd的配置。

# /etc/rancher/k3s/registries.yaml
mirrors:
  "docker.io":
    endpoint:
      - "https://fhnbkhe7.mirror.aliyuncs.com"
      - "https://registry-1.docker.io"

# 对于 private registry
mirrors:
  "192.168.8.45:5050":
    endpoint:
      - "http://192.168.8.45:5050"
  "*":
    endpoint:
      - "http://localhost:5000"

可以在 /var/lib/rancher/k3s/agent/etc/containerd/config.toml 下找到containerd相关的配置。

问题：k3s 中containerd相关的配置在什么地方？

另外的，关于如何使用私有镜像的教程： https://www.cnblogs.com/yaopengfei/p/13705822.html。

(注意的是，阿里云的私有镜像，在界面上要使用旧版的才能进行文中进行的操作。)

uninstall #

server node run

/usr/local/bin/k3s-uninstall.sh

worker node run

/usr/local/bin/k3s-agent-uninstall.sh

containerd in k3s #

configure image registry in k3s

containerd default configuration locates at /etc/containerd/config.toml.

k3s containerd unix socket: /run/k3s/containerd/containerd.sock.

Starting from 1.22, k3s passes through CONTAINERD_ environmental variables to containerd. One can add CONTAINERD_HTTP_PROXY, CONTAINERD_HTTPS_PROXY, CONTAINERD_NO_PROXY to systemd env file to set up proxy for pulling image. PR