调整前的kubelet配置

[root@dev-istio-master-001 ~]# cat /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf 
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
[root@dev-istio-master-001 ~]# cat /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1
[root@dev-istio-master-001 ~]# cat /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS=

其中:

  • kubelet命令有4个定义启动参数的变量,$KUBELET_KUBECONFIG_ARGS、$KUBELET_CONFIG_ARGS在服务的配置文件中定义,$KUBELET_KUBEADM_ARGS在/var/lib/kubelet/kubeadm-flags.env文件中定义,$KUBELET_EXTRA_ARGS在/etc/sysconfig/kubelet文件中定义。$KUBELET_EXTRA_ARGS参数优先级最高,用户自定义的配置应该在该文件中修改;
  • --bootstrap-kubeconfig:用于节点加入集群时,如果--kubeconfig指定的文件不存在,则kubelet使用--bootstrap-kubeconfig指定的文件中的用户名和token向kube-apiserver发送TLS Bootstrapping请求,然后将认证信息写入--kubeconfig指定的文件,证书写入--cert-dir指定的目录(默认/var/lib/kubelet/pki");
  • --kubeconfig:kubelet连接kube-apiserver的相关信息,节点第一次加入集群时自动生成该文件;
  • --config:kubelet主要的参数配置文件;
  • --cgroup-driver:操作宿主机cgroup的驱动;
  • --network-plugin:网络插件,这里使用CNI插件;
  • --pod-infra-container-image:Pause容器的镜像。

kubelet主配置文件config.yaml

[root@dev-istio-master-001 ~]# cat /var/lib/kubelet/config.yaml
address: 0.0.0.0
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
cgroupDriver: cgroupfs
cgroupsPerQOS: true
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
configMapAndSecretChangeDetectionStrategy: Watch
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuCFSQuotaPeriod: 100ms
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebuggingHandlers: true
enforceNodeAllocatable:
- pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:
  imagefs.available: 15%
  memory.available: 100Mi
  nodefs.available: 10%
  nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
fileCheckFrequency: 20s
hairpinMode: promiscuous-bridge
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 20s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
kind: KubeletConfiguration
kubeAPIBurst: 10
kubeAPIQPS: 5
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
nodeLeaseDurationSeconds: 40
nodeStatusReportFrequency: 1m0s
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
port: 10250
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
volumeStatsAggPeriod: 1m0s

其中:

  • address:kubelet API监听地址,不能为127.0.0.1,否则kube-apiserver、heapster等不能调用kubelet的API;
  • authentication.anonymous.enabled: false:禁止匿名访问kubelet;
  • authentication.webhook.enabled: true:开启HTTPs bearer token认证;
  • authentication.x509.clientCAFile:指定签名客户端证书的CA证书,开启HTTPS证书认证;
  • clusterDNS:集群DNS服务器的Cluster IP地址;
  • clusterDomain:集群域名后缀,默认为cluster.local,此时cloud名字空间test服务的FQDN为test.cloud.svc.cluster.local
  • containerLogMaxFiles: 5:单个容器保存的日志文件数量;
  • containerLogMaxSize: 10Mi:容器日志文件的大小达到10M滚动日志;
  • evictionHard.imagefs.available: 15%: 镜像文件系统空间少于15%时Pod被强行驱逐;
  • evictionHard.memory.available: 100Mi:节点可用内存少于100M时Pod被强行驱逐,修改为1Gi增加系统的稳定性;
  • evictionHard.nodefs.available: 10%:节点容器所用文件系统空间少于10%时Pod被强行驱逐;
  • evictionHard.nodefs.inodesFree: 5%:节点容器所用文件系统可用i-节点数量少于5%时Pod被强行驱逐;
  • port:kubelet API监听的端口;
  • nodeStatusReportFrequency:向kube-apiserver报告节点状态的频率;
  • healthzBindAddress:健康检查使用的ip地址;
  • healthzPort:健康检查使用的端口;
  • maxPods: 110:节点能运行的最大Pod数量;
  • staticPodPath: /etc/kubernetes/manifests:静态Pod的配置文件目录,容器中部署kubernetes时kube-apiserver、scheduler等的yaml文件存放位置。

参考文档

  • No labels