Commit graph

133 commits

Author SHA1 Message Date
Etienne Champetier
067db686f6
Fix proxy usage when *_PROXY are present in environment (#7309)
Since a790935d02 all proxy users
should be properly configured

Now when you have *_PROXY vars in your environment it can leads to failure
if NO_PROXY is not correct, or to persistent configuration changes
as seen with kubeadm in 1c5391dda7

Instead of playing constant whack-a-bug, inject empty *_PROXY vars everywhere
at the play level, and override at the task level when needed

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
2021-02-23 09:44:02 -08:00
Etienne Champetier
1c5391dda7
Ensure kubeadm doesn't use proxy (#7275)
* Move proxy_env to kubespray-defaults/defaults

There is no reasons to use set_facts here

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>

* Ensure kubeadm doesn't use proxy

*_proxy variables might be present in the environment (/etc/environment, bash profile, ...)
When this is the case we end up with those proxy configuration in /etc/kubernetes/manifests/kube-*.yaml manifests

We cannot unset env variables, but kubeadm is nice enough to ignore empty vars
93d288e2a4/cmd/kubeadm/app/util/env.go (L27)

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
2021-02-16 08:44:58 -08:00
Etienne Champetier
de1d9df787
Only use stat get_checksum: yes when needed (#7270)
By default Ansible stat module compute checksum, list extended attributes and find mime type
To find all stat invocations that really use one of those:
git grep -F stat. | grep -vE 'stat.(islnk|exists|lnk_source|writeable)'

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
2021-02-10 05:36:59 -08:00
Florian Ruynat
91ee4aa542
Decrease docker dependency (#7172) 2021-01-18 01:41:44 -08:00
Kenichi Omichi
c005c90746
Remove unnecessary failed_when (#7120)
TASK [Generate a list of information about the images on a node]
registers list of container images to docker_images.
Then the next TASK [Set pull_required if the desired image is not
yet loaded] does based on expecting images are registered.
However sometimes the first TASK was failed as [1] but the failure
is ignored due to failed_when:false and it makes another issue.
This removes this unnecessary failed_when to detect the failure
at the point.
In addition, this removes no_log:true also because the output doesn't
contain any sensitive data and now it just makes debugging difficult.

[1]: https://gitlab.com/kargo-ci/kubernetes-sigs-kubespray/-/jobs/934714534#L2953
2021-01-11 08:49:10 -08:00
Etienne Champetier
a790935d02
Only setup *_PROXY env variables where needed (#7095)
no_proxy is a pain to get right, and having proxy variables present causes issues
(k8s components get proxy configuration after upgrade, see #7100)

It's better to only configure what require proxy:
- the runtime (containerd/docker/crio)
- the package manager + apt_key
- the download tasks

Tested with the following clusters
- 4 CentOS 8 nodes
- 1 Ubuntu 20.04 node

Signed-off-by: Etienne Champetier <champetier.etienne@gmail.com>
2021-01-11 07:21:08 -08:00
Florian Ruynat
c971debd15
Fix crictl with Docker (#7081) 2020-12-23 08:28:26 -08:00
Sergey
096bcdd078
Download once for crio (#6998)
* download run once feature for CRI-O

* fix typo

* fix test
2020-12-21 01:54:25 -08:00
Sergey
a69f2b09da
download run once feature for containerd (#6997) 2020-12-07 01:09:25 -08:00
Hans Feldt
878fe80ca3
add and use common crictl role (#6978) 2020-12-05 09:43:25 -08:00
Hans Feldt
80eb1ad936
fix ansible password authentication (#6907)
* copying ssh key no longer required, works with password auth
* use copy module instead of synchronize (which requires sshpass)
* less tasks and always changed tasks
2020-11-30 15:12:50 -08:00
Mateus Caruccio
8908a70c19
Fails if kubeadm_version do not matches kubernetes version (#6302) 2020-09-21 07:20:32 -07:00
Florian Ruynat
151b142d30
Ignore pause from kubeadm config images list (#6689) 2020-09-18 07:32:46 -07:00
Barry Melbourne
058438a25d
Remove support for CoreOS Container Linux (#6576) 2020-08-28 02:28:53 -07:00
Maxime Guyot
6e938a3106
Fix E306 in other roles (#6517) 2020-08-28 01:20:53 -07:00
Maxime Guyot
214e08f8c9
Fix ansible-lint E305 (#6459) 2020-07-28 01:39:08 -07:00
Maxime Guyot
e70f27dd79
Add noqa and disable .ansible-lint global exclusions (#6410) 2020-07-27 06:24:17 -07:00
bozzo
276c450759
Use connection: local when delegate_to: localhost (#6322)
This will avoid SSH connection on the local host
2020-06-25 08:14:38 -07:00
Sergey
8f3d820664
always download docker image on download_host when download_run_once=true (#5921) 2020-04-10 01:59:47 -07:00
nmr
d152dc2e6a
Update prep_download.yml (#5791)
Fix check if user can use docker without sudo.
2020-03-18 13:30:44 -07:00
Sergey
03d1b56a8f
fix check exists download cache (#5776) 2020-03-16 03:34:35 -07:00
Kubernetes Prow Robot
66408a87ee
Refactor download role (#5697)
* download file

* download containers

* fix push image to nodes

* pull if none image on host

* fix

* improve docker image tag checks.
do not pull already cached images

* rebase fix merge conflict

* add support download_run_once when upgrade and scale cluster
add some test with download_run_once

* set default values to temp flag for every download cycle

* add save,load abilty for containerd and crio when download_run_once=true

* return redefine image save/load command to  set_docker_image_facts.yml

* move set command to set_container_facts

* ctr in containerd_bin_dir

* fix order of ctr image export arguments

* temporary disable download_run_once for containerd and crio
due https://github.com/containerd/containerd/issues/4075

* remove unused files

* fix strict yaml linter warning and errors

* refactor logical conditions to pull and cache container images

* remove comment due lint check

* document role

* remove image_load_on_localhost, because cached images are always loaded to docker on remote sites

* remove XXX from debug output
2020-03-05 07:31:39 -08:00
Moritz Graf
d2c44dd4df
Modifying Ansible filter 'failed' according to Ansible 2.5 porting guide (#5678) 2020-02-26 00:18:26 -08:00
Sylvain Chateau
0ca7aa126b
added "Flatcar", "Flatcar Container Linux by Kinvolk" for all coreOS role (#5607) 2020-02-18 00:15:29 -08:00
Maxime Guyot
b15d41a96a Add support to Ansible 2.9 (#5361) 2019-12-05 07:24:32 -08:00
Matthew Mosesohn
bd014c409b Skip coredns image when evaluating kubeadm images (#5327)
It will be enabled correctly in downloads

Change-Id: Ief0b7aa2a8ee2ba6a6849820802f8542584b2c04
Related-story: PRODX-1171
2019-11-09 00:51:39 -08:00
Sergey
65e461a7c0 download container always been on download_delegate host (#5177)
* download container always been on download_delegate host

* fix also check pull required
2019-10-17 05:38:38 -07:00
陈谭军
8bc0710073 clean up document (#5214) 2019-10-02 04:41:07 -07:00
Matthew Mosesohn
a43e0d3f95 Switch to Kubernetes v1.16.0 (#5189)
* Switch to Kubernetes v1.16.0

Change-Id: I5d6a9528b2d443750fc5e031aff15ad3ffead158

* Fix download localhost cached file path

Change-Id: I65e79b70e3d1b37265ebc60f41b460cf4b0a0d47

* fix kubeadm etcd for v1.16

Change-Id: I6888a00fd48b530a38b0b31c4095492476af42d2

* disable tf packet jobs

Change-Id: I075c4666547fdea4c50ec04864f38e2cfaa79154

* Disable contiv packet jobs. Fix kube-router

Change-Id: I3170e8789e60711d4cee8faf65f2094480b79b8d

* bump sonobuoy version

Change-Id: Ib946905629c7c53ed88f08fb2f41c454457a0097
2019-10-02 02:21:07 -07:00
陈谭军
99dbc6d780 clean-up doc,spelling mistakes (#5206) 2019-09-26 04:25:08 -07:00
Matthew Mosesohn
eb40ac163f Move cri_socket var to kubespray-defaults (#5149) 2019-09-10 12:30:55 -07:00
Florent Monbillard
637f09f140 Fix ansible task titles (#5154)
* Fix ansible task titles for CRI connection tasks

* Fix Azure subscription ID check task title
2019-09-10 01:34:54 -07:00
Sergey
494a6512b8 fix bug: run Copy image to ansible host cache on download_delegate host (#5094)
* run 'task download_container | Copy image to ansible host cache' with synchronize on download_delegate host

* try to run task copy file to ansible host on all inventory, not only on first random host
2019-08-21 23:38:30 -07:00
刘旭
de9443a694 remove unused code (#4981) 2019-07-16 01:39:24 -07:00
Waret
781b5691c9 prep_kubeadm_images: parse repo and tag (#4976) (#4977) 2019-07-15 03:15:09 -07:00
Sergey Kolekonov
46bef931e9 Fix images info logic for containerd (#4965)
As crictl tool is used to download images, it must be also used to gather
images info
2019-07-12 03:29:07 -07:00
Matthew Mosesohn
29307740dd Enable containerd to deploy vanilla containerd package (#4951)
* Enable containerd to deploy vanilla containerd package

Fixes kubeadm references to CRI socket for containerd
Fixes download role cache feature to work with containerd

Change-Id: I2ab8f0031107e2f0d1a85c39b4beb66f08509a01

* use containerd for flannel-addons job

Change-Id: Ied375c7d65e64a625ffbd995ff16f2374067dee6

* add containerd vars

Change-Id: Ib9a8a04e501c481a86235413cbec63f3672baf91

* fixup vars

Change-Id: Ibea64e4b18405a578b52a13da100384582aa24c2

* more fixes

* fix rh repo

Change-Id: I00575a77cfb7b81d6095db5d918a52023c8f13ba

* Adjust helm host install for containerd
2019-07-10 23:46:54 -07:00
Matthew Mosesohn
65065e7fdf Default kubeadm_images to empty dict instead of set_fact (#4957)
Change-Id: I9ef52232176fe8e2a99b4f09ae05e1451f7ad1ff
2019-07-09 12:07:30 -07:00
okamototk
4c8b93e5b9 containerd support (#4664)
* Add limited containerd support

Containerd support for Ubuntu + Calico

* Added CRI-O support for ubuntu

* containerd support.

* Reset  containerd support.

* fix lint.

* implemented feedback

* Change task name cri xx instead of cri-o in reset task and timeout condition.

* set crictl to fixed version

* Use docker-ce's container.io package for containerd.

* Add check containerd is installable or not.

* Avoid stop docker when use containerd and optimize retry for reset.

* Add config.toml.

* Fixed containerd for kubelet.env.

* Merge PR #4629

* Remove unused ubuntu variable for containerd

* Polish code for containerd and cri-o

* Refactoring cri socket configuration.

* Configurable conmon.

* Remove unused crictl/runc download

* Now crictl and runc is downloaded by common crictl.yml.

* fixed yamllint error

* Fixed brokenfiles by conflict.

* Remove commented line in config.toml

* Remove readded v1.12.x version

* Fixed broken set_docker_image_facts

* Fix yamllint errors.

* Remove unused apt source

* Fix crictl could not be installed

* Add containerd config from skolekonov's PR #4601
2019-06-29 14:09:20 -07:00
Matthew Mosesohn
6025981ceb Allow skip kubeadm image prep but install kubeadm (#4904)
Change-Id: I744e9a192cd863a1ce22fbd16d217c5dfb16750c
2019-06-23 23:17:56 -07:00
Johnny Halfmoon
23c9071c30 Added file and container image caching (#4828)
* File and container image downloads are now cached localy, so that repeated vagrant up/down runs do not trigger downloading of those files. This is especially useful on laptops with kubernetes runnig locally on vm's. The total size of the cache, after an ansible run, is currently around 800MB, so bandwidth (=time) savings can be quite significant.

* When download_run_once is false, the default is still not to cache, but setting download_force_cache will still enable caching.

* The local cache location can be set with download_cache_dir and defaults to /tmp/kubernetes_cache

* A local docker instance is no longer required to cache docker images; Images are cached to file. A local docker instance is still required, though, if you wish to download images on localhost.

* Fixed a FIXME, wher the argument was that delegate_to doesn't play nice with omit. That is a correct observation and the fix is to use default(inventory_host) instead of default(omit). See ansible/ansible#26009

* Removed "Register docker images info" task from download_container and set_docker_image_facts because it was faulty and unused.

* Removed redundant when:download.{container,enabled,run_once} conditions from {sync,download}_container.yml

* All features of commit d6fd0d2aca by Timoses <timosesu@gmail.com>, merged May 1st 2019, are included in this patch. Not all code was included verbatim, but each feature of that commit was checked to be working in this patch. One notable change: The actual downloading of the kubeadm images was moved to {download,sync)_container, to enable caching.

Note 1: I considered splitting this patch, but most changes that are not directly related to caching, are a pleasant by-product of implementing the caching code, so splitting would be impractical.

Note 2: I have my doubts about the usefulness of the upload, download and upgrade tags in the download role. Must they remain or can they be removed? If anybody knows, then please speak up.
2019-06-10 11:21:07 -07:00
Matthew Mosesohn
3617ae31f6 Optionally skip predownload of kubeadm images (#4832) 2019-06-04 04:35:02 -07:00
MarkusTeufelberger
73c2ff17dd Fix Ansible-lint error [E502] (#4743) 2019-05-16 00:27:43 -07:00
Timoses
13f225e6ae Only pull images for destined host groups (#4735)
Without this, pulls are considered for all
hosts groups, even if not targetted by the downloads
`groups` list. Hence, a download/sync is triggered
even though the host does not require the image.
2019-05-16 00:25:48 -07:00
MarkusTeufelberger
e67f848abc ansible-lint: add spaces around variables [E206] (#4699) 2019-05-02 14:24:21 -07:00
Maxime Guyot
f29387316f Fix ansible-lint 602 (#4688) 2019-05-01 23:42:17 -07:00
Timoses
d6fd0d2aca Enable delegating all downloads (binaries, images, kubeadm images) (#4420)
* Download to delegate and sync files when download_run_once

* Fail on error after saving container image

* Do not set changed status when downloaded container was up to date

* Only sync containers when they are actually required

Previously, non-required images (pull_required=false as
image existed on target host) were synced to the target
hosts. This failed as the image was not downloaded to
the download_delegate and hence was not available for
syncing.

* Sync containers when only missing on some hosts

* Consider images with multiple repo tags

* Enable kubeadm images pull/syncing with download_delegate

* Use kubeadm images list to pull/sync

'kubeadm config images pull' is replaced by collecting the images
list with 'kubeadm config images list' and using the commonly
used method of pull/syncing the images.

* Ensure containers are downloaded and synced for all hosts

* Fix download/syncing when download_delegate is a kubernetes host
2019-05-01 01:10:56 -07:00
MarkusTeufelberger
88d919337e ansible-lint: don't compare to empty string [E602] (#4665) 2019-04-28 23:00:20 -07:00
iwankgb
4e81bcc147 Fixing Vagrant cluster provisioning (#4218)
* Pass ansible_ssh_user as host_var

Co-authored-by: Damian Darczuk <damian.darczuk@intel.com>
Co-authored-by: Paweł Pałucki <pawel.palucki@intel.com>

* Create a directory before downloading container images to ansible host

Co-authored-by: Damian Darczuk <damian.darczuk@intel.com>
Co-authored-by: Paweł Pałucki <pawel.palucki@intel.com>

* Set private key usuing synchronize task options

Co-authored-by: Damian Darczuk <damian.darczuk@intel.com>
Co-authored-by: Paweł Pałucki <pawel.palucki@intel.com>
2019-04-24 05:42:05 -07:00
MarkusTeufelberger
d2a1ac3b0c Add Ansible-lint CI step (#4411)
* Add ansible-lint as gitlab-ci step

* Fix jinja2 syntax in include_tasks that breaks ansible-lint

* Use a block scalar to get around gitlab quoting/escaping rules

* Run ansible-lint in verbose mode in CI
2019-04-10 02:04:16 -07:00