diff --git a/README.md b/README.md index 94676bb7c..950ce225f 100644 --- a/README.md +++ b/README.md @@ -16,8 +16,8 @@ Linux distributions tested: ### Requirements * The target servers must have **access to the Internet** in order to pull docker imaqes. -* The firewalls are not managed, you'll need to implement your own rules the way you used to. -in order to avoid any issue during deployment you should **disable your firewall** +* The **firewalls are not managed**, you'll need to implement your own rules the way you used to. +in order to avoid any issue during deployment you should disable your firewall * **Copy your ssh keys** to all the servers part of your inventory. * **Ansible v2.x and python-netaddr** * Base knowledge on Ansible. Please refer to [Ansible documentation](http://www.ansible.com/how-ansible-works) @@ -37,16 +37,20 @@ These defaults are good for tests purposes. Edit the inventory according to the number of servers ``` [kube-master] -10.115.99.31 +node1 +node2 [etcd] -10.115.99.31 -10.115.99.32 -10.115.99.33 +node1 +node2 +node3 [kube-node] -10.115.99.32 -10.115.99.33 +node2 +node3 +node4 +node5 +node6 [k8s-cluster:children] kube-node @@ -68,37 +72,32 @@ The main variables to change are located in the directory ```inventory/group_var ### Inventory Below is an example of an inventory. -Note : The bgp vars local_as and peers are not mandatory if the var **'peer_with_router'** is set to false -By default this variable is set to false and therefore all the nodes are configure in **'node-mesh'** mode. -In node-mesh mode the nodes peers with all the nodes in order to exchange routes. ``` +## Configure 'ip' variable to bind kubernetes services on a +## different ip than the default iface +node1 ansible_ssh_host=95.54.0.12 # ip=10.3.0.1 +node2 ansible_ssh_host=95.54.0.13 # ip=10.3.0.2 +node3 ansible_ssh_host=95.54.0.14 # ip=10.3.0.3 +node4 ansible_ssh_host=95.54.0.15 # ip=10.3.0.4 +node5 ansible_ssh_host=95.54.0.16 # ip=10.3.0.5 +node6 ansible_ssh_host=95.54.0.17 # ip=10.3.0.6 [kube-master] -node1 ansible_ssh_host=10.99.0.26 -node2 ansible_ssh_host=10.99.0.27 +node1 +node2 [etcd] -node1 ansible_ssh_host=10.99.0.26 -node2 ansible_ssh_host=10.99.0.27 -node3 ansible_ssh_host=10.99.0.4 +node1 +node2 +node3 [kube-node] -node2 ansible_ssh_host=10.99.0.27 -node3 ansible_ssh_host=10.99.0.4 -node4 ansible_ssh_host=10.99.0.5 -node5 ansible_ssh_host=10.99.0.36 -node6 ansible_ssh_host=10.99.0.37 - -[paris] -node1 ansible_ssh_host=10.99.0.26 -node3 ansible_ssh_host=10.99.0.4 local_as=xxxxxxxx -node4 ansible_ssh_host=10.99.0.5 local_as=xxxxxxxx - -[new-york] -node2 ansible_ssh_host=10.99.0.27 -node5 ansible_ssh_host=10.99.0.36 local_as=xxxxxxxx -node6 ansible_ssh_host=10.99.0.37 local_as=xxxxxxxx +node2 +node3 +node4 +node5 +node6 [k8s-cluster:children] kube-node @@ -138,8 +137,6 @@ Kubernetes * You can choose where to install the master components. If you want your master node to act both as master (api,scheduler,controller) and node (e.g. accept workloads, create pods ...), the server address has to be present on both groups 'kube-master' and 'kube-node'. -* Almost all kubernetes components are running into pods except *kubelet*. These pods are managed by kubelet which ensure they're always running - * For safety reasons, you should have at least two master nodes and 3 etcd servers * Kube-proxy doesn't support multiple apiservers on startup ([Issue 18174](https://github.com/kubernetes/kubernetes/issues/18174)). An external loadbalancer needs to be configured. @@ -155,16 +152,6 @@ You can choose between 2 network plugins. Only one must be chosen. The choice is defined with the variable '**kube_network_plugin**' -### Expose a service -There are several loadbalancing solutions. -The one i found suitable for kubernetes are [Vulcand](http://vulcand.io/) and [Haproxy](http://www.haproxy.org/) - -My cluster is working with haproxy and kubernetes services are configured with the loadbalancing type '**nodePort**'. -eg: each node opens the same tcp port and forwards the traffic to the target pod wherever it is located. - -Then Haproxy can be configured to request kubernetes's api in order to loadbalance on the proper tcp port on the nodes. - -Please refer to the proper kubernetes documentation on [Services](https://github.com/kubernetes/kubernetes/blob/release-1.0/docs/user-guide/services.md) ### Check cluster status @@ -199,7 +186,7 @@ There are two ways of installing new apps Additionnal apps can be installed with ```ansible-galaxy```. -ou'll need to edit the file '*requirements.yml*' in order to chose needed apps. +you'll need to edit the file '*requirements.yml*' in order to chose needed apps. The list of available apps are available [there](https://github.com/ansibl8s) For instance it is **strongly recommanded** to install a dns server which resolves kubernetes service names. @@ -268,6 +255,18 @@ calicoctl pool show calicoctl endpoint show --detail ``` +##### Optionnal : BGP Peering with border routers + +In some cases you may want to route the pods subnet and so NAT is not needed on the nodes. +For instance if you have a cluster spread on different locations and you want your pods to talk each other no matter where they are located. +The following variables need to be set: +**peer_with_router** enable the peering with border router of the datacenter (default value: false). +you'll need to edit the inventory and add a and a hostvar **local_as** by node. +``` +node1 ansible_ssh_host=95.54.0.12 local_as=xxxxxx +``` + + #### Flannel * Flannel configuration file should have been created there diff --git a/requirements.yml b/requirements.yml index b90b1715d..d89dbe72d 100644 --- a/requirements.yml +++ b/requirements.yml @@ -1,39 +1,30 @@ --- - src: https://github.com/ansibl8s/k8s-common.git path: roles/apps - version: v1.0 - src: https://github.com/ansibl8s/k8s-kubedns.git path: roles/apps - version: v1.0 #- src: https://github.com/ansibl8s/k8s-kube-ui.git # path: roles/apps -# version: v1.0 # #- src: https://github.com/ansibl8s/k8s-fabric8.git # path: roles/apps -# version: v1.0 # #- src: https://github.com/ansibl8s/k8s-elasticsearch.git # path: roles/apps -# # version: v1.0 # #- src: https://github.com/ansibl8s/k8s-redis.git # path: roles/apps -# # version: v1.0 # #- src: https://github.com/ansibl8s/k8s-memcached.git # path: roles/apps -# version: v1.0 # #- src: https://github.com/ansibl8s/k8s-postgres.git # path: roles/apps -# version: v1.0 # #- src: https://github.com/ansibl8s/k8s-pgbouncer.git # path: roles/apps -# version: v1.0 # #- src: https://github.com/ansibl8s/k8s-heapster.git # path: roles/apps @@ -43,3 +34,6 @@ # #- src: https://github.com/ansibl8s/k8s-kubedash.git # path: roles/apps +# +#- src: https://github.com/ansibl8s/k8s-kube-logstash.git +# path: roles/apps diff --git a/roles/apps/k8s-elasticsearch b/roles/apps/k8s-elasticsearch index 3d74c70a4..f089f60fb 160000 --- a/roles/apps/k8s-elasticsearch +++ b/roles/apps/k8s-elasticsearch @@ -1 +1 @@ -Subproject commit 3d74c70a4a2d294ec86528468efa2c10ece7bd7b +Subproject commit f089f60fb1102378b6def3972b50644deff96484 diff --git a/roles/apps/k8s-fabric8 b/roles/apps/k8s-fabric8 index 82ca8293b..702923e20 160000 --- a/roles/apps/k8s-fabric8 +++ b/roles/apps/k8s-fabric8 @@ -1 +1 @@ -Subproject commit 82ca8293b091a3e6141f435eb1186bdd7b912bf2 +Subproject commit 702923e2000d07bb95044c747c499bb04c3d16bf diff --git a/roles/apps/k8s-heapster b/roles/apps/k8s-heapster index 44a6519bf..5442209ac 160000 --- a/roles/apps/k8s-heapster +++ b/roles/apps/k8s-heapster @@ -1 +1 @@ -Subproject commit 44a6519bf8957bff316d3e3bc857d554f69c4016 +Subproject commit 5442209acd072d170f9d24e2bbd00b95af737d7d diff --git a/roles/apps/k8s-influxdb b/roles/apps/k8s-influxdb index 38d54c48e..b892974d5 160000 --- a/roles/apps/k8s-influxdb +++ b/roles/apps/k8s-influxdb @@ -1 +1 @@ -Subproject commit 38d54c48e76df2951d60a6e17a40d5a3d69a1e81 +Subproject commit b892974d5b8bc624ac742e8a6fb7737db8a96a32 diff --git a/roles/apps/k8s-kube-logstash b/roles/apps/k8s-kube-logstash index 256fa156e..c1c0ce8e2 160000 --- a/roles/apps/k8s-kube-logstash +++ b/roles/apps/k8s-kube-logstash @@ -1 +1 @@ -Subproject commit 256fa156e46d623ab0a7a60efdc7bac535cea8d7 +Subproject commit c1c0ce8e27b430cff8e5b0f1519707eb892f5e67 diff --git a/roles/apps/k8s-kube-ui b/roles/apps/k8s-kube-ui index b81a2848d..0b5be08de 160000 --- a/roles/apps/k8s-kube-ui +++ b/roles/apps/k8s-kube-ui @@ -1 +1 @@ -Subproject commit b81a2848d9e0be5f65c61dcb902b63fc60475633 +Subproject commit 0b5be08de2b3f373146a36aef88be3d4e5565dc2 diff --git a/roles/apps/k8s-kubedash b/roles/apps/k8s-kubedash index 64385696a..21544cc2a 160000 --- a/roles/apps/k8s-kubedash +++ b/roles/apps/k8s-kubedash @@ -1 +1 @@ -Subproject commit 64385696a907f8052cd455b85302fbd2d0bab129 +Subproject commit 21544cc2a908b1578409c5eaca7ee2771b3b9811 diff --git a/roles/apps/k8s-kubedns b/roles/apps/k8s-kubedns index d6df09a89..82b75f841 160000 --- a/roles/apps/k8s-kubedns +++ b/roles/apps/k8s-kubedns @@ -1 +1 @@ -Subproject commit d6df09a89721d98e2969a8abf29b4eb5e787fca6 +Subproject commit 82b75f84129189fcfb08adfeb6ed6114c68dde3e diff --git a/roles/apps/k8s-memcached b/roles/apps/k8s-memcached index a5bdd1d71..60a66ca0a 160000 --- a/roles/apps/k8s-memcached +++ b/roles/apps/k8s-memcached @@ -1 +1 @@ -Subproject commit a5bdd1d7141af57074e76878323a27410fc449ce +Subproject commit 60a66ca0ae4dc9a7e29b9b9619134adada53e1a4 diff --git a/roles/apps/k8s-pgbouncer b/roles/apps/k8s-pgbouncer index 22f9618b3..79961d1df 160000 --- a/roles/apps/k8s-pgbouncer +++ b/roles/apps/k8s-pgbouncer @@ -1 +1 @@ -Subproject commit 22f9618b3209538e64fb2d8f030b45119bc5a9f2 +Subproject commit 79961d1df223dd1563e41a1df109c4cffa8801ed diff --git a/roles/apps/k8s-rabbitmq b/roles/apps/k8s-rabbitmq index b91f96bb9..2b538e243 160000 --- a/roles/apps/k8s-rabbitmq +++ b/roles/apps/k8s-rabbitmq @@ -1 +1 @@ -Subproject commit b91f96bb9cff2dbd1614a090263893b35862f3d1 +Subproject commit 2b538e243a3230c97591a28ac438619ebd555e4c