Commit graph

496 commits

Author SHA1 Message Date
Matthew Mosesohn
bf3425322f Significantly reduce memory requirements
Canal runs more pods and upgrades need a bit of extra
room to load new pods in and get the old ones out.
2017-03-27 13:28:37 +03:00
Matthew Mosesohn
f1f58166e5 Only cordon Ready nodes 2017-03-27 13:28:37 +03:00
Matthew Mosesohn
986a89be66 Merge pull request #1181 from holser/refactor_etcd
Refactor etcd role
2017-03-27 13:05:35 +03:00
Sergii Golovatiuk
77671dbd05 Refactor etcd role
- Run docker run from script rather than directly from systemd target
- Refactoring styling/templates

Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
2017-03-24 12:34:15 +01:00
Vladimir Rutsky
bc30ea7582 replace non-breakable space with regular space
Non-brekable space is 0xc2 0xa0 byte sequence in UTF-8.

To find one:

    $ git grep -I -P '\xc2\xa0'

To replace with regular space:

    $ git grep -l -I -P '\xc2\xa0' | xargs sed -i 's/\xc2\xa0/ /g'

This commit doesn't include changes that will overlap with commit f1c59a91a1.
2017-03-23 00:25:01 +03:00
Matthew Mosesohn
51985ab9de Change wait for dnsmasq to skip if there are no kube-nodes in play
Also changed unnecessary delay to a max timeout (now defaulting to 1s sleep
between tries)

Also rename play_hosts to ansible_play_hosts
2017-03-21 18:55:22 +03:00
Matthew Mosesohn
182a2e682f Merge pull request #1149 from mattymo/centos-retries
Retry yum/apt/rpm download commands
2017-03-18 11:12:36 +03:00
Matthew Mosesohn
dd56342361 Retry yum/apt/rpm download commands, fix succeeded filter 2017-03-17 18:56:26 +03:00
Matthew Mosesohn
b060d3279c Merge pull request #1146 from mattymo/resolvconf_optimize
Condense resolvconf sources before starting loop
2017-03-17 18:42:32 +03:00
Matthew Mosesohn
a5876a1ac8 Merge pull request #1136 from adidenko/fix-calico-policy-order
Move calico-policy-controller into separate role
2017-03-17 17:32:14 +03:00
Aleksandr Didenko
1adf231512 Move calico-policy-controller into separate role
By default Calico CNI does not create any network access policies
or profiles if 'policy' is enabled in CNI config. And without any
policies/profiles network access to/from PODs is blocked.

K8s related policies are created by calico-policy-controller in
such case. So we need to start it as soon as possible, before any
real workloads.

This patch also fixes kube-api port in calico-policy-controller
yaml template.

Closes #1132
2017-03-17 11:21:52 +01:00
Matthew Mosesohn
2586d01345 Condense resolvconf sources before starting loop 2017-03-17 13:06:56 +03:00
Matthew Mosesohn
bf5bf13003 Merge pull request #1087 from bradbeam/openstack
Adding openstack domain id
2017-03-16 17:53:14 +03:00
Matthew Mosesohn
ad03f3ac84 Merge branch 'master' into idempotency2 2017-03-16 09:29:43 +03:00
Matthew Mosesohn
261aeb6112 Merge pull request #1138 from mattymo/idempotency-fixes
Idempotency fixes for etcd certs and resolvconf tasks
2017-03-16 09:20:28 +03:00
Matthew Mosesohn
fad22bae97 More idempotency fixes
Fixed sync_tokens fact
Fixed sync_certs for k8s tokens fact
Disabled register docker images changability
Fixed CNI dir permission
Fix idempotency for etcd pre upgrade checks
2017-03-15 19:06:39 +03:00
Matthew Mosesohn
20247b9c0a Merge pull request #1080 from VincentS/Granular_Auth_Control
Granular authentication Control
2017-03-15 13:12:51 +03:00
Matthew Mosesohn
210d9503f3 Merge pull request #1117 from mattymo/etcd3-upgrade
Migrate k8s data to etcd3 api store
2017-03-15 12:56:06 +03:00
Matthew Mosesohn
4287993811 Make resolvconf preinstall idempotent 2017-03-15 01:20:13 +04:00
Vincent Schwarzer
ea1f072c7e Granular authentication Control
It is now possible to deactivate selected authentication methods
(basic auth, token auth) inside the cluster by adding
removing the required arguments to the Kube API Server and generating
the secrets accordingly.

The x509 authentification is currently not optional because disabling it
would affect the kubectl clients deployed on the master nodes.
2017-03-14 16:57:35 +01:00
Matthew Mosesohn
8450ff00cc Merge pull request #1134 from mattymo/1.6-support
Explicitly set cni-bin-dir
2017-03-14 17:53:08 +03:00
Matthew Mosesohn
25b366dd98 Migrate k8s data to etcd3 api store
Default backend is now etcd3 (was etcd2).
The migration process consists of the following steps:
* check if migration is necessary
* stop etcd on first etcd server
* run migration script
* start etcd on first etcd server
* stop kube-apiserver until configuration is updated
* update kube-apiserver
* purge old etcdv2 data
2017-03-14 17:50:20 +03:00
Matthew Mosesohn
e486dabc42 Merge pull request #1078 from VincentS/oidc_support
Added Support for OpenID Connect Authentication
2017-03-14 12:07:21 +03:00
Matthew Mosesohn
944fa9d975 Explicitly set cni-bin-dir 2017-03-13 20:13:21 +03:00
Matthew Mosesohn
f2900d65e1 Merge pull request #1118 from mattymo/noderolelabels
Add node labels in kubelet
2017-03-13 19:04:21 +03:00
Cesarini, Daniele
d7f5fdc0c5 Adding /O=system:masters to admin certificate
Issue #1125. Make RBAC authorization plugin work out of the box.
"When bootstrapping, superuser credentials should include the system:masters group, for example by creating a client cert with /O=system:masters. This gives those credentials full access to the API and allows an admin to then set up bindings for other users."
2017-03-08 14:42:25 +00:00
Brad Beam
0c96b5d3fc Removing cloud_provider tag to fix scenario where cloud_provider is not defined 2017-03-06 10:52:38 -06:00
Matthew Mosesohn
8065a2355c Add node labels in kubelet
Related-issue: https://github.com/kubernetes/community/issues/300
Upgraded nodes do not obtain labels automatically.
See https://github.com/kubernetes/kubernetes/pull/29459 for more details.
2017-03-06 17:18:42 +03:00
Vincent Schwarzer
ea6bf9143f Added Support for OpenID Connect Authentication
To use OpenID Connect Authentication beside deploying an OpenID Connect
Identity Provider it is necesarry to pass additional arguments to the Kube API Server.
These required arguments were added to the kube apiserver manifest.
2017-03-06 12:40:35 +01:00
Antoine Legrand
bfe58a7750 Merge pull request #1045 from bradbeam/vsphere
Adding vsphere cloud provider support
2017-03-06 12:34:05 +01:00
Antoine Legrand
8595bf50cd Merge pull request #1112 from mattymo/skip_vault_if_disabled
Disable vault role properly on ansible 2.2.0
2017-03-06 11:27:53 +01:00
Matthew Mosesohn
7a3956173a Disable vault role properly on ansible 2.2.0
when condition does not seem to work correctly at playbook
level for ansible 2.2.0.
2017-03-05 00:43:01 +04:00
Matthew Mosesohn
cd3c402454 Merge pull request #1111 from mattymo/use_find_for_certs
Use find module for checking for certificates
2017-03-03 20:08:33 +03:00
Matthew Mosesohn
7e1aa3b43b Use find module for checking for certificates
Also generate certs only when absent on master (rather than
when absent on target node)
2017-03-03 16:21:01 +03:00
Bogdan Dobrelya
1cca1909c9 Merge pull request #1071 from vijaykatam/atomic_host
Add support for atomic host
2017-03-03 13:03:59 +01:00
Matthew Mosesohn
4b50274b33 Merge pull request #1075 from VincentS/loadbalancer_aws
Possibility to add Loadbalancers without static IP (e.g. AWS ELB) #1074
2017-03-03 14:07:22 +03:00
Matthew Mosesohn
d1299f358b Merge pull request #1060 from holser/etcdv3
Allow to specify etcd backend for kube-api
2017-03-02 17:24:09 +03:00
Vincent Schwarzer
9164b9cdcf Changes based on feedback (additional ansible checks) 2017-03-02 11:04:10 +01:00
Vincent Schwarzer
3ef7365cae Modified how adding LB for the Kube API is handled (AWS)
Until now it was not possible to add an API Loadbalancer
without an static IP Address. But certain Loadbalancers
like AWS Elastic Loadbalanacer dontt have an fixed IP address.
With this commit it is possible to add these kind of Loadbalancers
to the Kargo deployment.
2017-03-02 11:04:10 +01:00
Matthew Mosesohn
b6861ebed0 Merge pull request #959 from galthaus/host-mode-restart
Restart kube-controller for host_resolvconf mode
2017-03-01 20:54:21 +03:00
Vijay Katam
8fc5a844b3 Add support for atomic host
Updates based on feedback

Simplify checks for file exists

remove invalid char

Review feedback. Use regular systemd file.

Add template for docker systemd atomic
2017-03-01 09:38:19 -08:00
Sergii Golovatiuk
d9f67a343c Allow to specify etcd backend for kube-api
Kubernetes project is about to set etcdv3 as default storage engine in
1.6. This patch allows to specify particular backend for
kube-apiserver. User may force the option to etcdv3 for new environment.
At the same time if the environment uses v2 it will continue uses it
until user decides to upgrade to v3.

Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
2017-02-28 17:13:22 +01:00
Brad Beam
a939c53d86 Adding KUBELET_CLOUDPROVIDER to kubelet.rkt.service 2017-02-28 06:29:35 -06:00
Brad Beam
607fb7c89d Making openstack domain name optional 2017-02-27 21:19:27 -06:00
Xavier Lange
60af40af27 Bug fix: support kilo's keystone requirement for domain-name, extracts from ENV var 2017-02-27 21:18:30 -06:00
Brad Beam
6a144213c9 Updating vsphere cloud provider support 2017-02-27 15:08:04 -06:00
Jan Jungnickel
c11f981692 Initial support for vsphere as cloud provider 2017-02-27 12:51:41 -06:00
Brad Beam
c50bb7d252 Lower default memory requests
This is to address out of memory issues on CI as well as help
fit deployments for people starting out with kargo on smaller
machines
2017-02-27 10:53:43 -06:00
Bogdan Dobrelya
543dafa900 Merge pull request #1063 from bogdando/fix
Align LB defaults with the HA docs
2017-02-27 10:14:42 +01:00
Sergii Golovatiuk
802503458d Increase SSL TTL to 3650 days
In real scenarios 365 days is short period of time. 3650 days is good
enough for long running k8s environments
2017-02-24 15:38:13 +01:00
Bogdan Dobrelya
18cb160be6 Align LB defaults with the HA docs
Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru>
2017-02-23 10:32:44 +01:00
Bogdan Dobrelya
dacfbde0b0 Rework inventory all by real groups' vars
* Leave all.yml to keep only optional vars
* Store groups' specific vars by existing group names
* Fix optional vars casted as mandatory (add default())
* Fix missing defaults for an optional IP var
* Relink group_vars for terraform to reflect changes

Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru>
2017-02-23 09:43:42 +01:00
Matthew Mosesohn
9a1774dfd7 Merge pull request #1020 from mattymo/synthscale
Add synthetic scale deployment mode
2017-02-22 19:15:46 +03:00
Ivan Shvedunov
836a6a953a Fix shell special vars 2017-02-21 22:22:40 +03:00
Matthew Mosesohn
51a6ec836e Merge branch 'master' into synthscale 2017-02-21 22:17:43 +03:00
Matthew Mosesohn
efe8f20af8 Merge pull request #1046 from skyscooby/pedantic-syntax-cleanup
Cleanup legacy syntax, spacing, files all to yml
2017-02-21 17:03:16 +03:00
Matthew Mosesohn
967df623f9 Merge pull request #1034 from rutsky/fix-openssl-lb-index
fix load balancer DNS name index evaluation in openssl.conf
2017-02-20 20:23:26 +03:00
Matthew Mosesohn
c2b05783c9 Merge branch 'master' into pedantic-syntax-cleanup 2017-02-20 20:19:38 +03:00
Abel Lopez
22efbd0153 Safe disable SELinux
Sometimes, a sysadmin might outright delete the SELinux rpms and
delete the configuration. This causes the selinux module to fail
with
```
IOError: [Errno 2] No such file or directory: '/etc/selinux/config'\n",
"module_stdout": "", "msg": "MODULE FAILURE"}
```

This simply checks that /etc/selinux/config exists before we try
to set it Permissive.

Update from feedback
2017-02-18 11:54:25 -08:00
Matthew Mosesohn
5419c98eb1 Add no_log to cert tar tasks
This works around 4MB limit for gitlab CI runner.
2017-02-18 14:09:57 +04:00
Andrew Greenwood
756003a30e Cleanup legacy syntax, spacing, files all to yml
Migrate older inline= syntax to pure yml syntax for module args as to be consistant with most of the rest of the tasks
Cleanup some spacing in various files
Rename some files named yaml to yml for consistancy
2017-02-17 16:22:34 -05:00
Antoine Legrand
1862098948 Merge pull request #1038 from rutsky/kubelet-mount-var-log
Mount host's /var/log into kubelet container
2017-02-17 10:26:12 +01:00
Vladimir Rutsky
0089bf5b9d Mount host's /var/log into kubelet container
Kubelet is responsible for creating symlinks from /var/lib/docker to /var/log
to make fluentd logging collector work.
However without using host's /var/log those links are invisible to fluentd.

This is done on rkt configuration too.
2017-02-16 22:31:05 +03:00
Matthew Mosesohn
02736d1ff0 Fix references to CoreOS and Container Linux by CoreOS
Fixes #967
2017-02-16 19:25:17 +03:00
Vladimir Rutsky
9af63e173d fix load balancer DNS name index evaluation in openssl.conf
Looks like OpenSSL still properly handles it, even with duplicated
"DNS.X" items.
2017-02-16 00:16:13 +03:00
Matthew Mosesohn
f12fa9d8b5 Merge pull request #985 from rutsky/check-mode-for-shell-commands
set "check_mode: on" for read-only "shell" steps that registers result
2017-02-15 17:53:41 +03:00
Brad Beam
2a3dee5f90 Adding support for proxy w/ rkt kubelet 2017-02-14 08:09:49 -06:00
Matthew Mosesohn
68f441fbdb Merge pull request #1013 from mattymo/remove_masqerade_all
Disable kube_proxy_masquerade_all
2017-02-14 14:00:29 +03:00
Matthew Mosesohn
73a8613e42 Merge pull request #1021 from holser/remove_deprecated
Replace always_run with check_mode
2017-02-14 11:25:58 +03:00
Vladimir Rutsky
fff8780a51 set "check_mode: no" for read-only "shell" steps that registers result
"shell" step doesn't support check mode, which currently leads to failures,
when Ansible is being run in check mode (because Ansible doesn't run command,
assuming that command might have effect, and no "rc" or "output" is registered).

Setting "check_mode: no" allows to run those "shell" commands in check mode
(which is safe, because those shell commands doesn't have side effects).
2017-02-13 18:53:41 +03:00
Greg Althaus
932629cc38 When resolv.conf changes during host_resolvconf mode, we need to
restart the controller to get the new file configuration.
I'm not fond of this form and would like a better way, but this
seems to "work".
2017-02-13 09:20:02 -06:00
Sergii Golovatiuk
959517aa62 Replace always_run with check_mode
always_run was deprecated in Ansible 2.2 and will be removed in 2.4
ansible logs contain "[DEPRECATION WARNING]: always_run is deprecated.
Use check_mode = no instead". This patch fix deprecation.
2017-02-13 15:00:56 +01:00
Sergii Golovatiuk
5494d608e5 Set ssl_ca_dirs for rkt based on fact
Since systemd kubelet.service has {{ ssl_ca_dirs }}, fact should be
gathered before writing kubelet.service.

Closes: #1007
Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
2017-02-13 13:28:29 +01:00
Matthew Mosesohn
21d648ad36 Disable kube_proxy_masquerade_all
Fixes #1012
2017-02-10 13:16:39 +03:00
Bogdan Dobrelya
0ddcc74412 Merge pull request #1002 from code0x9/master
use ansible sysctl module for config ip forwarding
2017-02-10 10:40:18 +01:00
Matthew Mosesohn
14e10988fc Merge pull request #989 from holser/kubelet_remedy
Kubernetes Reliability Improvements
2017-02-10 09:29:29 +03:00
Sergii Golovatiuk
c0d2cca45d Kubernetes Reliability Improvements
- Exclude kubelet CPU/RAM (kube-reserved) from cgroup. It decreases a
  chance of overcommitment
- Add a possibility to modify Kubelet node-status-update-frequency
- Add a posibility to configure node-monitor-grace-period,
  node-monitor-period, pod-eviction-timeout for Kubernetes controller
  manager
- Add Kubernetes Relaibility Documentation with recomendations for
  various scenarios.

Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
2017-02-09 23:54:08 +01:00
Greg Althaus
d6f1f0c88b Make kubelet_load_modules always present but false.
Update code and docs for that assumption.
2017-02-09 10:25:44 -06:00
Greg Althaus
8f00a07bf6 Due to the nsenter and other reworks, it appears that
kubelet lost the ability to load kernel modules.  This
puts that back by adding the lib/modules mount to kubelet.

The new variable kubelet_load_modules can be set to true
to enable this item.  It is OFF by default.
2017-02-09 10:02:26 -06:00
Mark Lee
331957c9a8 follow sysctl.conf file symlink if linked 2017-02-09 18:16:52 +09:00
Mark Lee
224e6acb3a use ansible sysctl module for config ip forwarding 2017-02-09 17:28:44 +09:00
Josh Conant
764ad6e099 Vault security hardening and role isolation 2017-02-08 21:41:36 +00:00
Matthew Mosesohn
ce3bee4eb8 Merge pull request #990 from mattymo/fix_cert_upgrade
Fix check for node-NODEID certs existence
2017-02-08 14:44:09 +03:00
Matthew Mosesohn
3c7952d7f1 Merge pull request #992 from vwfs/host_mount_dev
Host mount /dev for kubelet
2017-02-08 13:45:22 +03:00
Bogdan Dobrelya
320d03c01c Merge pull request #956 from adidenko/update-netchecker
Update playbooks to support new netchecker
2017-02-08 10:09:46 +01:00
Alexander Block
08367f4abb Host mount /dev for kubelet 2017-02-08 09:55:51 +01:00
Matthew Mosesohn
012bc49404 Fix check for node-NODEID certs existence
Fixes upgrade from pre-individual node cert envs.
2017-02-07 21:06:48 +03:00
Aleksandr Didenko
3b816ee660 Update playbooks to support new netchecker
Netchecker is rewritten in Go lang with some new args instead of
env variables. Also netchecker-server no longer requires kubectl
container. Updating playbooks accordingly.
2017-02-07 15:20:34 +01:00
Matthew Mosesohn
7a9161d462 Prevent dynamic port allocation in nodePort range
kube_apiserver_node_port_range should be accessible only
to kube-proxy and not be taken by a dynamic port allocation.

Potentially temporary if https://github.com/kubernetes/kubernetes/issues/40920
gets fixed.
2017-02-06 20:01:16 +03:00
Matthew Mosesohn
be24fab5d0 Revert "Drop linux capabilities and rework users/groups" 2017-02-06 15:58:54 +03:00
Bogdan Dobrelya
9d31173db4 Merge pull request #911 from bogdando/DROP_CAPS
Drop linux capabilities and rework users/groups
2017-02-06 12:05:51 +01:00
Bogdan Dobrelya
e53d3fe9c8 Merge pull request #949 from vmtyler/master
Fixes Support for OpenStack v3 credentials
2017-02-03 12:22:00 +01:00
Matthew Mosesohn
b7cbd5b9c5 Merge pull request #927 from holser/nsenter_fix
Remove nsenter workaround
2017-02-02 18:18:15 +03:00
Sergii Golovatiuk
b610c1627e Remove nsenter workaround
- Docker 1.12 and further don't need nsenter hack. This patch removes
  it.  Also, it bumps the minimal version to 1.12.

Closes #776

Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
2017-02-02 14:38:11 +01:00
Greg Althaus
0037d0e6b2 This continues the DHCP hook checks. Also protect the create side
if the system doesn't have any config files at all.
2017-01-31 09:56:27 -06:00
Matthew Mosesohn
1011e416a7 Fix cert distribution at scale
Use stdin instead of bash args to pass node filenames and base64 data.
Use tempfile for master cert data
2017-01-31 16:27:45 +03:00
Tyler Britten
6b29c6c702 Fixed for non-null output 2017-01-27 10:47:59 -05:00
Tyler Britten
ec1c47bc5a Updated OpenStack vars to check for tenant_id (v2) and project_id (v3) 2017-01-27 10:26:20 -05:00
Xavier Lange
eb07363ddb Bugfix: skip cloud_config on etcd 2017-01-25 14:09:21 -08:00
Bogdan Dobrelya
48e77cd8bb Drop linux capabilities and rework users/groups
* Drop linux capabilities for unprivileged containerized
  worlkoads Kargo configures for deployments.
* Configure required securityContext/user/group/groups for kube
  components' static manifests, etcd, calico-rr and k8s apps,
  like dnsmasq daemonset.
* Rework cloud-init (etcd) users creation for CoreOS.
* Fix nologin paths, adjust defaults for addusers role and ensure
  supplementary groups membership added for users.
* Add netplug user for network plugins (yet unused by privileged
  networking containers though).
* Grant the kube and netplug users read access for etcd certs via
  the etcd certs group.
* Grant group read access to kube certs via the kube cert group.
* Remove priveleged mode for calico-rr and run it under its uid/gid
  and supplementary etcd_cert group.
* Adjust docs.
* Align cpu/memory limits and dropped caps with added rkt support
  for control plane.

Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru>
2017-01-20 08:50:42 +01:00