k8s nodes is forbidden user cannot list resource nodes in api group at the cluster scope
继续将k8s用于模型转换和部署的自动化流程...然后发现之前安装k8s的文档不work了．． 时间是2020年5月7日，当前最新的k8s版本是 v1.18.2
报错如下:
1
2
3<2kzzqw6rsjid0 --discovery-token-ca-cert-hash sha256:c6c72bdc96c0ff4d59559ff915eee61ba7ac5e8b93c0b2f9e11e813412387ec2 --v=5
4W0507 15:45:12.608784 4768 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
5I0507 15:45:12.608822 4768 join.go:371] [preflight] found NodeName empty; using OS hostname as NodeName
6I0507 15:45:12.608853 4768 initconfiguration.go:103] detected and using CRI socket: /var/run/dockershim.sock
7[preflight] Running pre-flight checks
8I0507 15:45:12.608902 4768 preflight.go:90] [preflight] Running general checks
9I0507 15:45:12.608933 4768 checks.go:249] validating the existence and emptiness of directory /etc/kubernetes/manifests
10I0507 15:45:12.608966 4768 checks.go:286] validating the existence of file /etc/kubernetes/kubelet.conf
11I0507 15:45:12.608975 4768 checks.go:286] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
12I0507 15:45:12.608985 4768 checks.go:102] validating the container runtime
13I0507 15:45:12.685381 4768 checks.go:128] validating if the service is enabled and active
14 [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
15I0507 15:45:12.765669 4768 checks.go:335] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
16I0507 15:45:12.765720 4768 checks.go:335] validating the contents of file /proc/sys/net/ipv4/ip_forward
17I0507 15:45:12.765752 4768 checks.go:649] validating whether swap is enabled or not
18I0507 15:45:12.765780 4768 checks.go:376] validating the presence of executable conntrack
19I0507 15:45:12.765804 4768 checks.go:376] validating the presence of executable ip
20I0507 15:45:12.765826 4768 checks.go:376] validating the presence of executable iptables
21I0507 15:45:12.765844 4768 checks.go:376] validating the presence of executable mount
22I0507 15:45:12.765864 4768 checks.go:376] validating the presence of executable nsenter
23I0507 15:45:12.765882 4768 checks.go:376] validating the presence of executable ebtables
24I0507 15:45:12.765902 4768 checks.go:376] validating the presence of executable ethtool
25I0507 15:45:12.765920 4768 checks.go:376] validating the presence of executable socat
26I0507 15:45:12.765935 4768 checks.go:376] validating the presence of executable tc
27I0507 15:45:12.765953 4768 checks.go:376] validating the presence of executable touch
28I0507 15:45:12.765973 4768 checks.go:520] running all checks
29I0507 15:45:12.844881 4768 checks.go:406] checking whether the given node name is reachable using net.LookupHost
30I0507 15:45:12.845030 4768 checks.go:618] validating kubelet version
31I0507 15:45:12.888056 4768 checks.go:128] validating if the service is enabled and active
32I0507 15:45:12.893254 4768 checks.go:201] validating availability of port 10250
33I0507 15:45:12.893373 4768 checks.go:286] validating the existence of file /etc/kubernetes/pki/ca.crt
34I0507 15:45:12.893388 4768 checks.go:432] validating if the connectivity type is via proxy or direct
35I0507 15:45:12.893414 4768 join.go:441] [preflight] Discovering cluster-info
36I0507 15:45:12.893440 4768 token.go:78] [discovery] Created cluster-info discovery client, requesting info from "172.20.52.117:6443"
37I0507 15:45:13.033539 4768 token.go:116] [discovery] Requesting info from "172.20.52.117:6443" again to validate TLS against the pinned public key
38I0507 15:45:13.172634 4768 token.go:133] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "172.20.52.117:6443"
39I0507 15:45:13.172653 4768 discovery.go:51] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
40I0507 15:45:13.172660 4768 join.go:455] [preflight] Fetching init configuration
41I0507 15:45:13.172669 4768 join.go:493] [preflight] Retrieving KubeConfig objects
42[preflight] Reading configuration from the cluster...
43[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
44I0507 15:45:13.541858 4768 interface.go:400] Looking for default routes with IPv4 addresses
45I0507 15:45:13.541871 4768 interface.go:405] Default route transits interface "eth0"
46I0507 15:45:13.541941 4768 interface.go:208] Interface eth0 is up
47I0507 15:45:13.541978 4768 interface.go:256] Interface "eth0" has 2 addresses :[10.198.21.97/22 fe80::f816:3eff:fe5e:88f1/64].
48I0507 15:45:13.541998 4768 interface.go:223] Checking addr 10.198.21.97/22.
49I0507 15:45:13.542008 4768 interface.go:230] IP found 10.198.21.97
50I0507 15:45:13.542016 4768 interface.go:262] Found valid IPv4 address 10.198.21.97 for interface "eth0".
51I0507 15:45:13.542023 4768 interface.go:411] Found active IP 10.198.21.97
52I0507 15:45:13.542057 4768 preflight.go:101] [preflight] Running configuration dependant checks
53I0507 15:45:13.542072 4768 controlplaneprepare.go:211] [download-certs] Skipping certs download
54I0507 15:45:13.542080 4768 kubelet.go:111] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
55I0507 15:45:13.542775 4768 kubelet.go:119] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
56I0507 15:45:13.543283 4768 kubelet.go:145] [kubelet-start] Checking for an existing Node in the cluster with name "host-10-198-21-97" and status "Ready"
57nodes "host-10-198-21-97" is forbidden: User "system:bootstrap:0752yx" cannot get resource "nodes" in API group "" at the cluster scope
58cannot get Node "host-10-198-21-97"
59k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/join.runKubeletStartJoinPhase
60 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/join/kubelet.go:148
61k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
62 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:234
63k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
64 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:422
65k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
66 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:207
67k8s.io/kubernetes/cmd/kubeadm/app/cmd.NewCmdJoin.func1
68 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/join.go:170
69k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute
70 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:826
71k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC
72 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:914
73k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute
74 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:864
75k8s.io/kubernetes/cmd/kubeadm/app.Run
76 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/kubeadm.go:50
77main.main
78 _output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
79runtime.main
80 /usr/local/go/src/runtime/proc.go:203
81runtime.goexit
82 /usr/local/go/src/runtime/asm_amd64.s:1357
83error execution phase kubelet-start
84k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
85 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:235
86k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
87 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:422
88k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
89 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:207
90k8s.io/kubernetes/cmd/kubeadm/app/cmd.NewCmdJoin.func1
91 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/join.go:170
92k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute
93 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:826
94k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC
95 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:914
96k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute
97 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:864
98k8s.io/kubernetes/cmd/kubeadm/app.Run
99 /workspace/anago-v1.18.2-beta.0.14+a78cd082e8c913/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/kubeadm.go:50
100main.main
101 _output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
102runtime.main
103 /usr/local/go/src/runtime/proc.go:203
104runtime.goexit
105 /usr/local/go/src/runtime/asm_amd64.s:1357
106
看起来重点的报错在这一句
1
2nodes "host-10-198-21-97" is forbidden: User "system:bootstrap:0752yx" cannot get resource "nodes" in API group "" at the cluster scope
3cannot get Node "host-10-198-21-97"
4
然后google发现大概是权限相关的原因...Role-Based Access Contro 相关的． 但是似乎都不是在搭建集群的时候遇到的．
然后打算重新看一遍最新的搭建手册，发现troubleshooting里面
Not possible to join a v1.18 Node to a v1.17 cluster due to missing RBAC
原来是v1.18增加了权限控制，1.18的slave机器没办法加入到1.17的master节点上...看来就是这个问题．
然后在控制节点上apply了如下内容:
1
2apiVersion: rbac.authorization.k8s.io/v1
3kind: ClusterRole
4metadata:
5 name: kubeadm:get-nodes
6rules:
7- apiGroups:
8 - ""
9 resources:
10 - nodes
11 verbs:
12 - get
13---
14apiVersion: rbac.authorization.k8s.io/v1
15kind: ClusterRoleBinding
16metadata:
17 name: kubeadm:get-nodes
18roleRef:
19 apiGroup: rbac.authorization.k8s.io
20 kind: ClusterRole
21 name: kubeadm:get-nodes
22subjects:
23- apiGroup: rbac.authorization.k8s.io
24 kind: Group
25 name: system:bootstrappers:kubeadm:default-node-token
26
重新尝试加入，已经可以了．