Kube-router =========== Kube-router is a L3 CNI provider, as such it will setup IPv4 routing between nodes to provide Pods' networks reachability. See [kube-router documentation](https://www.kube-router.io/). ## Verifying kube-router install Kube-router runs its pods as a `DaemonSet` in the `kube-system` namespace: * Check the status of kube-router pods ``` # From the CLI kubectl get pod --namespace=kube-system -l k8s-app=kube-router -owide # output NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE kube-router-4f679 1/1 Running 0 2d 192.168.186.4 mykube-k8s-node-nf-2 kube-router-5slf8 1/1 Running 0 2d 192.168.186.11 mykube-k8s-node-nf-3 kube-router-lb6k2 1/1 Running 0 20h 192.168.186.14 mykube-k8s-node-nf-6 kube-router-rzvrb 1/1 Running 0 20h 192.168.186.17 mykube-k8s-node-nf-4 kube-router-v6n56 1/1 Running 0 2d 192.168.186.6 mykube-k8s-node-nf-1 kube-router-wwhg8 1/1 Running 0 20h 192.168.186.16 mykube-k8s-node-nf-5 kube-router-x2xs7 1/1 Running 0 2d 192.168.186.10 mykube-k8s-master-1 ``` * Peek at kube-router container logs: ``` # From the CLI kubectl logs --namespace=kube-system -l k8s-app=kube-router | grep Peer.Up # output time="2018-09-17T16:47:14Z" level=info msg="Peer Up" Key=192.168.186.6 State=BGP_FSM_OPENCONFIRM Topic=Peer time="2018-09-17T16:47:16Z" level=info msg="Peer Up" Key=192.168.186.11 State=BGP_FSM_OPENCONFIRM Topic=Peer time="2018-09-17T16:47:46Z" level=info msg="Peer Up" Key=192.168.186.10 State=BGP_FSM_OPENCONFIRM Topic=Peer time="2018-09-18T19:12:24Z" level=info msg="Peer Up" Key=192.168.186.14 State=BGP_FSM_OPENCONFIRM Topic=Peer time="2018-09-18T19:12:28Z" level=info msg="Peer Up" Key=192.168.186.17 State=BGP_FSM_OPENCONFIRM Topic=Peer time="2018-09-18T19:12:38Z" level=info msg="Peer Up" Key=192.168.186.16 State=BGP_FSM_OPENCONFIRM Topic=Peer [...] ``` ## Gathering kube-router state Kube-router Pods come bundled with a "Pod Toolbox" which provides very useful internal state views for: * IPVS: via `ipvsadm` * BGP peering and routing info: via `gobgp` You need to `kubectl exec -it ...` into a kube-router container to use these, see for details. ## Kube-router configuration You can change the default configuration by overriding `kube_router_...` variables (as found at `roles/network_plugin/kube-router/defaults/main.yml`), these are named to follow `kube-router` command-line options as per . ## Caveats ### kubeadm_enabled: true If you want to set `kube-router` to replace `kube-proxy` (`--run-service-proxy=true`) while using `kubeadm_enabled`, then 'kube-proxy` DaemonSet will be removed *after* kubeadm finishes running, as it's not possible to skip kube-proxy install in kubeadm flags and/or config, see https://github.com/kubernetes/kubeadm/issues/776. Given above, if `--run-service-proxy=true` is needed it would be better to void `kubeadm_enabled` i.e. set: ``` kubeadm_enabled: false kube_router_run_service_proxy: true ``` If for some reason you do want/need to set `kubeadm_enabled`, removing it afterwards behave better if kube-proxy is set to ipvs mode, i.e. set: ``` kubeadm_enabled: true kube_router_run_service_proxy: true kube_proxy_mode: ipvs ``` ## Advanced BGP Capabilities https://github.com/cloudnativelabs/kube-router#advanced-bgp-capabilities If you have other networking devices or SDN systems that talk BGP, kube-router will fit in perfectly. From a simple full node-to-node mesh to per-node peering configurations, most routing needs can be attained. The configuration is Kubernetes native (annotations) just like the rest of kube-router. For more details please refer to the https://github.com/cloudnativelabs/kube-router/blob/master/docs/bgp.md. Next options will set up annotations for kube-router, using `kubectl annotate` command. ``` kube_router_annotations_master: [] kube_router_annotations_node: [] kube_router_annotations_all: [] ```