Chains of iptables used by Kubernetes
Several chains of iptables are programed to do all kinds of filtering and NAT between pods and services when a Service or Endpoint object is created.
KUBE-SERVICESis the entry point for service packets. What it does is to match the destination IP:port and dispatch the packet to the corresponding KUBE-SVC-* chain.KUBE-SVC-*acts as a load balancer, which distributes the packet to KUBE-SEP-* chain. The number of KUBE-SEP-* is equal to the number of endpoints behind the service. Which KUBE-SEP-* to be chosen is determined randomly.KUBE-SEP-*represents a Service EndPoint. It simply does DNAT, replacing service IP:port with pod’s endpoint IP:Port.- See: https://serenafeng.github.io/2020/03/26/kube-proxy-in-iptables-mode/
Traefik Status
- Traefik loadbalancer sould be already configured for following ports:
- 8087 -> keycloak Service
- 80 -> HTTP access
- 443 -> HTTPS access
- 8080 -> Traefik Admin port
- The loadbalancer should route any traefik packages to the traefik loadbalancer running on ip 10.43.99.34
- The svclb-traefik_XXX pods are runnng an iptables command to trigger this routing
- The svclb-traefik_XXX pods are runing on each cluster node
- Each svclb-traefik_XXX pod runs 4 containers [ lb-port-80, lb-port-443, lb-port-8080, lb-port-8087 ]
- Traefik pod is listening on pod 8087
# kubectl get pod -n traefik-ns -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES svclb-traefik-8rl5h 4/4 Running 0 23h 10.42.0.163 kube-master <none> <none> svclb-traefik-nds69 4/4 Running 0 23h 10.42.2.246 kube-worker2 <none> <none> traefik-59cfdc49c7-dmm58 1/1 Running 0 23h 10.42.2.245 kube-worker2 <none> <none> svclb-traefik-mk54q 4/4 Running 0 23h 10.42.1.31 kube-worker1 <none> <none> # kubectl get svc -n traefik-ns -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR traefik LoadBalancer 10.43.99.34 141.75.148.138,141.75.148.139,141.75.148.140 80:31898/TCP,8080:32676/TCP,8087:30158/TCP,443:31879/TCP 27h app=traefik # kubectl get endpoints -n traefik-ns -o wide NAME ENDPOINTS AGE traefik 10.42.2.245:80,10.42.2.245:8087,10.42.2.245:443 + 1 more... 27h Validate Traefik Pod status # kubectl exec --stdin --tty traefik-59cfdc49c7-dmm58 -n traefik-ns sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. / # netstat -tulpen | egrep '8087 |State' Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 :::8087 :::* LISTEN 1/traefik
Package routing via iptables to traefik IP:Port
- URL: kube-master.informatik.fh-nuernberg.de:8087
- iptables routing to keycloak pod running kubeworker2 wiht IP:Port: 10.42.2.245:8087
root@kube-master:/home/master# iptables -t nat -L KUBE-SERVICES -n | column -t | egrep 'KUBE-SERVICES|traefik' Chain KUBE-SERVICES (2 references) KUBE-MARK-MASQ tcp -- !10.42.0.0/16 10.43.99.34 /* traefik-ns/traefik:admin cluster IP */ tcp dpt:8080 KUBE-SVC-2NGTPF5PU2OO4NFC tcp -- 0.0.0.0/0 10.43.99.34 /* traefik-ns/traefik:admin cluster IP */ tcp dpt:8080 KUBE-FW-2NGTPF5PU2OO4NFC tcp -- 0.0.0.0/0 141.75.148.138 /* traefik-ns/traefik:admin loadbalancer IP */ tcp dpt:8080 KUBE-FW-2NGTPF5PU2OO4NFC tcp -- 0.0.0.0/0 141.75.148.139 /* traefik-ns/traefik:admin loadbalancer IP */ tcp dpt:8080 KUBE-FW-2NGTPF5PU2OO4NFC tcp -- 0.0.0.0/0 141.75.148.140 /* traefik-ns/traefik:admin loadbalancer IP */ tcp dpt:8080 KUBE-MARK-MASQ tcp -- !10.42.0.0/16 10.43.99.34 /* traefik-ns/traefik:websecure cluster IP */ tcp dpt:443 KUBE-SVC-CATTP5HEIU3VUDDR tcp -- 0.0.0.0/0 10.43.99.34 /* traefik-ns/traefik:websecure cluster IP */ tcp dpt:443 KUBE-FW-CATTP5HEIU3VUDDR tcp -- 0.0.0.0/0 141.75.148.138 /* traefik-ns/traefik:websecure loadbalancer IP */ tcp dpt:443 KUBE-FW-CATTP5HEIU3VUDDR tcp -- 0.0.0.0/0 141.75.148.139 /* traefik-ns/traefik:websecure loadbalancer IP */ tcp dpt:443 KUBE-FW-CATTP5HEIU3VUDDR tcp -- 0.0.0.0/0 141.75.148.140 /* traefik-ns/traefik:websecure loadbalancer IP */ tcp dpt:443 KUBE-MARK-MASQ tcp -- !10.42.0.0/16 10.43.99.34 /* traefik-ns/traefik:web cluster IP */ tcp dpt:80 KUBE-SVC-6R2DDDBNAG6WBID2 tcp -- 0.0.0.0/0 10.43.99.34 /* traefik-ns/traefik:web cluster IP */ tcp dpt:80 KUBE-FW-6R2DDDBNAG6WBID2 tcp -- 0.0.0.0/0 141.75.148.138 /* traefik-ns/traefik:web loadbalancer IP */ tcp dpt:80 KUBE-FW-6R2DDDBNAG6WBID2 tcp -- 0.0.0.0/0 141.75.148.139 /* traefik-ns/traefik:web loadbalancer IP */ tcp dpt:80 KUBE-FW-6R2DDDBNAG6WBID2 tcp -- 0.0.0.0/0 141.75.148.140 /* traefik-ns/traefik:web loadbalancer IP */ tcp dpt:80 KUBE-MARK-MASQ tcp -- !10.42.0.0/16 10.43.99.34 /* traefik-ns/traefik:keycloak cluster IP */ tcp dpt:8087 KUBE-SVC-O2EOQBSIC5NZRIQN tcp -- 0.0.0.0/0 10.43.99.34 /* traefik-ns/traefik:keycloak cluster IP */ tcp dpt:8087 KUBE-FW-O2EOQBSIC5NZRIQN tcp -- 0.0.0.0/0 141.75.148.138 /* traefik-ns/traefik:keycloak loadbalancer IP */ tcp dpt:8087 KUBE-FW-O2EOQBSIC5NZRIQN tcp -- 0.0.0.0/0 141.75.148.139 /* traefik-ns/traefik:keycloak loadbalancer IP */ tcp dpt:8087 KUBE-FW-O2EOQBSIC5NZRIQN tcp -- 0.0.0.0/0 141.75.148.140 /* traefik-ns/traefik:keycloak loadbalancer IP */ tcp dpt:8087 root@kube-master:/home/master# iptables -v -t nat -L KUBE-SVC-O2EOQBSIC5NZRIQN -n | column -t Chain KUBE-SVC-O2EOQBSIC5NZRIQN (5 references) pkts bytes target prot opt in out source destination 1 60 KUBE-SEP-AFDBSPUPSG7DXKN6 all -- * * 0.0.0.0/0 0.0.0.0/0 /* traefik-ns/traefik:keycloak */ root@kube-master:/home/master# iptables -v -t nat -L KUBE-SEP-AFDBSPUPSG7DXKN6 -n | column -t Chain KUBE-SEP-AFDBSPUPSG7DXKN6 (1 references) pkts bytes target prot opt in out source destination 0 0 KUBE-MARK-MASQ all -- * * 10.42.2.245 0.0.0.0/0 /* traefik-ns/traefik:keycloak */ 1 60 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* traefik-ns/traefik:keycloak */ tcp to:10.42.2.245:8087
Be First to Comment