Commit graph

140 commits

Author SHA1 Message Date
Timoses
13f225e6ae Only pull images for destined host groups (#4735)
Without this, pulls are considered for all
hosts groups, even if not targetted by the downloads
`groups` list. Hence, a download/sync is triggered
even though the host does not require the image.
2019-05-16 00:25:48 -07:00
MarkusTeufelberger
e67f848abc ansible-lint: add spaces around variables [E206] (#4699) 2019-05-02 14:24:21 -07:00
Maxime Guyot
f29387316f Fix ansible-lint 602 (#4688) 2019-05-01 23:42:17 -07:00
Timoses
d6fd0d2aca Enable delegating all downloads (binaries, images, kubeadm images) (#4420)
* Download to delegate and sync files when download_run_once

* Fail on error after saving container image

* Do not set changed status when downloaded container was up to date

* Only sync containers when they are actually required

Previously, non-required images (pull_required=false as
image existed on target host) were synced to the target
hosts. This failed as the image was not downloaded to
the download_delegate and hence was not available for
syncing.

* Sync containers when only missing on some hosts

* Consider images with multiple repo tags

* Enable kubeadm images pull/syncing with download_delegate

* Use kubeadm images list to pull/sync

'kubeadm config images pull' is replaced by collecting the images
list with 'kubeadm config images list' and using the commonly
used method of pull/syncing the images.

* Ensure containers are downloaded and synced for all hosts

* Fix download/syncing when download_delegate is a kubernetes host
2019-05-01 01:10:56 -07:00
MarkusTeufelberger
88d919337e ansible-lint: don't compare to empty string [E602] (#4665) 2019-04-28 23:00:20 -07:00
iwankgb
4e81bcc147 Fixing Vagrant cluster provisioning (#4218)
* Pass ansible_ssh_user as host_var

Co-authored-by: Damian Darczuk <damian.darczuk@intel.com>
Co-authored-by: Paweł Pałucki <pawel.palucki@intel.com>

* Create a directory before downloading container images to ansible host

Co-authored-by: Damian Darczuk <damian.darczuk@intel.com>
Co-authored-by: Paweł Pałucki <pawel.palucki@intel.com>

* Set private key usuing synchronize task options

Co-authored-by: Damian Darczuk <damian.darczuk@intel.com>
Co-authored-by: Paweł Pałucki <pawel.palucki@intel.com>
2019-04-24 05:42:05 -07:00
MarkusTeufelberger
d2a1ac3b0c Add Ansible-lint CI step (#4411)
* Add ansible-lint as gitlab-ci step

* Fix jinja2 syntax in include_tasks that breaks ansible-lint

* Use a block scalar to get around gitlab quoting/escaping rules

* Run ansible-lint in verbose mode in CI
2019-04-10 02:04:16 -07:00
Florent Monbillard
2054a98cf7 Run kubeadm and hyperkube outside of local_release_dir (#4098)
Addressing the discussion started in #4064, this PR moves kubeadm and
hyperkube binaries to /usr/local/bin before running them on the master
nodes.

It is to address the case where local_release_dir points to /tmp
(kubespray default) and /tmp is mounted with noexec mode, preventing
any binaries to be run in that partition.

In role "node", we still move kubeadm to bin_dir only on the worker
nodes.
2019-01-28 02:00:49 -08:00
rongzhang
8d0158ceeb Fix kubeadm config images pull
Supported by kubeadm v1.11
2019-01-28 14:42:55 +08:00
Manuel Cintron
7633e6d582 Added pass through parameters to enable basic auth for downloads 2019-01-08 19:36:13 -06:00
Rong Zhang
5834e609a6 Add scale master features (#3946)
* Add scale master features

* Add certificate management with kubeadm

* Add kubeadm kubeconfig

* Fix ymalroles error

* fix upgrade cluster fialed

* force update cert and keys when you reconfigure cluster
2018-12-27 23:27:27 -08:00
Rong Zhang
6362211860 Add images downloader to download roles (#3914)
* Add images downloader to download roles

* Use single jinja2 templates

* add kube_version to templates
2018-12-19 05:17:58 -08:00
Miao Zhou
a585318b1a Fix Sync Container Permission (#3752)
When `ansible_user` is not root, using `-b` option.
And with `download_run_once` and `download_localhost` set `true`.

Ansible will executes `container_download | upload container images to nodes` task.

It uses rsync to upload images to `/tmp/release/container/`, but the
`container` directory owned by `root`.
2018-11-26 07:00:34 -08:00
Erwan Miran
14c2df0418 Replace raw module with shell to avoid warning (#3652) 2018-11-06 11:07:11 -08:00
Erwan Miran
3b787123e3 Fix tasks to avoid ansible warning about raw module environment (#3545) 2018-10-20 07:13:54 -07:00
Antoine Legrand
2a3aa591e0
Download role (#3553)
* codestyle tests

* Download destination can be different than local_release_dir
2018-10-20 13:56:55 +02:00
Erwan Miran
7bec169d58 Fix ansible syntax to avoid ansible deprecation warnings (#3512)
* failed

* version_compare

* succeeded

* skipped

* success

* version_compare becomes version since ansible 2.5

* ansible minimal version updated in doc and spec

* last version_compare
2018-10-16 15:33:30 -07:00
Erwan Miran
dd5327ef9e Fix ansible syntax to avoid ansible warnings (#3499) 2018-10-11 00:45:00 -07:00
Anatoly Rugalev
8f85ea89fa Added download_validate_certs option which allows to disables SSL validation for file downloads 2018-09-21 11:51:17 +02:00
Takashi Okamoto
5ab8a712d9 Add download_container flag to avoid docker pull when use cri-o. 2018-08-28 01:24:26 +00:00
Takashi Okamoto
359009bb05 Download etcd and hyperkube binary. 2018-08-28 01:24:26 +00:00
Alexandru Bogdan Pica
e63bc65a9d Fix 2976
Fix failure when the container attribute is not set for a download
2018-07-08 13:36:47 +03:00
elementyang
c0935e161b fix template index out of range for pull images 2018-06-23 05:32:44 +08:00
Matthew Mosesohn
61791bbb3d Remove condition for docker pull when using download delegate 2018-04-12 19:01:13 +03:00
Atoms
6c954df636 move when condition to main.yml 2018-04-11 12:05:33 +03:00
Andreas Krüger
2c89a02db3 Only download container/file if host is in defined group (#2565)
* Only download container/file if host is in defined group

* Set correct when clause

* Fix last entries

* Update download groups
2018-03-30 22:40:01 -04:00
Brad Beam
da173615e4
Merge pull request #2048 from xizhibei/master
Fix: always only one container got synced after download
2018-01-29 16:01:11 -06:00
Matthew Mosesohn
dc6a17e092
Use include/import tasks (#2192)
import_tasks will consume far less memory, so it should be
used whenever it is compatible.
2018-01-29 14:37:48 +03:00
Matthew Mosesohn
b135bcb9d9 Split download container task for delegate and non-delegate modes (#2077)
Ansible cannot seem to handle omitting delegate_to since v2.4.0.0.

Possibly related: https://github.com/ansible/ansible/issues/30760
2017-12-14 16:45:54 +00:00
Xu Zhipei
66f38a1b31 fix: always only one docker image got synced after download 2017-12-12 09:51:03 +08:00
Spencer Smith
b27453d8d8 improved proxy support 2017-10-30 11:42:14 -04:00
Matthew Mosesohn
a52bc44f5a Fix broken CI jobs (#1854)
* Fix broken CI jobs

Adjust image and image_family scenarios for debian.
Checkout CI file for upgrades

* add debugging to file download

* Fix download for alternate playbooks

* Update ansible ssh args to force ssh user

* Update sync_container.yml
2017-10-25 11:45:54 +01:00
Flavio Percoco Premoli
5b08277ce4 Access dict item's value keys using .value (#1865) 2017-10-24 20:49:36 +01:00
Matthew Mosesohn
cea41a544e Use include instead of import tasks to support v2.3 (#1855)
Eventually 2.3 support will be dropped, so this is
a temporary change.
2017-10-23 13:56:03 +01:00
Matthew Mosesohn
fc9a65be2b Refactor downloads to use download role directly (#1824)
* Refactor downloads to use download role directly

Also disable fact delegation so download delegate works acros OSes.

* clean up bools and ansible_os_family conditionals
2017-10-19 09:17:11 +01:00
Matthew Mosesohn
eb0dcf6063 Improve proxy (#1771)
* Set no_proxy to all local ips

* Use proxy settings on all necessary tasks
2017-10-11 19:47:27 +01:00
pmontanari
764b1aa5f8 Force synchronize to use ssh_args so it works when using bastion
In case ssh.config is set to use bastion, synchronize needs to use it too.
2017-10-06 00:21:54 +02:00
Aivars Sterns
9c86da1403 Normalize tags in all places to prepare for tag fixing in future (#1739) 2017-10-05 08:43:04 +01:00
Deni Bertovic
64740249ab Adds tags for asserts (#1639) 2017-09-25 08:41:03 +01:00
Matthew Mosesohn
f2057dd43d Refactor downloads (#1642)
* Refactor downloads

Add prefixes to tasks (file vs container)
Remove some delegates
Clean up some conditions

* Update ansible.cfg
2017-09-09 23:32:12 +03:00
Brad Beam
8b151d12b9 Adding yamllinter to ci steps (#1556)
* Adding yaml linter to ci check

* Minor linting fixes from yamllint

* Changing CI to install python pkgs from requirements.txt

- adding in a secondary requirements.txt for tests
- moving yamllint to tests requirements
2017-08-24 12:09:52 +03:00
Sergii Golovatiuk
674b71b535 Ansible 2.3 support
- Fix when clauses in various places
- Update requirements.txt
- Fix README.md

Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
2017-04-26 15:22:10 +02:00
Matthew Mosesohn
7760c3e4aa Retry yum/apt/rpm download commands, fix succeeded filter 2017-03-17 18:56:26 +03:00
Matthew Mosesohn
a422ad0d50 More idempotency fixes
Fixed sync_tokens fact
Fixed sync_certs for k8s tokens fact
Disabled register docker images changability
Fixed CNI dir permission
Fix idempotency for etcd pre upgrade checks
2017-03-15 19:06:39 +03:00
Matthew Mosesohn
d821448e2f Merge branch 'master' into synthscale 2017-02-21 22:17:43 +03:00
Matthew Mosesohn
475a42767a Suppress logging for download image
This generates too much output and during upgrade scenarios
can bring us over the 4mb limit.
2017-02-18 19:10:26 +04:00
Andrew Greenwood
fd17c37feb Regex syntax changes in yml mode 2017-02-17 17:30:39 -05:00
Andrew Greenwood
ca9ea097df Cleanup legacy syntax, spacing, files all to yml
Migrate older inline= syntax to pure yml syntax for module args as to be consistant with most of the rest of the tasks
Cleanup some spacing in various files
Rename some files named yaml to yml for consistancy
2017-02-17 16:22:34 -05:00
Vladimir Rutsky
09847567ae set "check_mode: no" for read-only "shell" steps that registers result
"shell" step doesn't support check mode, which currently leads to failures,
when Ansible is being run in check mode (because Ansible doesn't run command,
assuming that command might have effect, and no "rc" or "output" is registered).

Setting "check_mode: no" allows to run those "shell" commands in check mode
(which is safe, because those shell commands doesn't have side effects).
2017-02-13 18:53:41 +03:00
Josh Conant
245e05ce61 Vault security hardening and role isolation 2017-02-08 21:41:36 +00:00
Matthew Mosesohn
d8ae50800a Work around escaping curly braces for docker inspect 2017-01-17 20:35:38 +03:00
Bogdan Dobrelya
5af2c42bde Better fix for different CoreOS os family facts
Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru>
2017-01-05 16:32:08 +01:00
Bogdan Dobrelya
f7447837c5 Rename CoreOS fact
Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru>
2017-01-05 14:02:29 +01:00
Bogdan Dobrelya
a56d9de502 Systemd units, limits, and bin path fixes
* Add restart for weave service unit
* Reuse docker_bin_dir everythere
* Limit systemd managed docker containers by CPU/RAM. Do not configure native
  systemd limits due to the lack of consensus in the kernel community
  requires out-of-tree kernel patches.

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-12-28 15:49:42 +01:00
Bogdan Dobrelya
79996b557b Rework ignore_errors to report no reds
Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru>
2016-12-27 13:00:50 +01:00
Bogdan Dobrelya
b8bc8eee41 Add download_always_pull check and sha256 for docker images
Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-12-20 17:02:09 +01:00
Bogdan Dobrelya
fd9b26675e More granular control for download/upload images/binaries
Add upload tag allow users to exclude distributing images across nodes
when running with the download tag set.
Add related tags and update docs as well.

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-12-09 17:04:55 +01:00
Bogdan Dobrelya
8cc84e132a Add tags
Add tags to allow more granular tasks filtering.
Add generator script for MD formatted tags found.
Add docs for tags how-to.

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-12-09 12:14:28 +01:00
Bogdan Dobrelya
aa447585c4 Fix download dnsmasq image dependency on docker
When download_run_once with download_localhost is used, docker is
expected to be running on the delegate localhost. That may be not
the case for a non localhost delegate, which is the kube-master
otherwise. Then the dnsmasq role, had it been invoked early before
deployment starts, would fail because of the missing docker dependency.

* Fix that dependency on docker and do not pre download dnsmasq image
  for the dnsmasq role, if download_localhost is disabled.
* Remove become: false for docker CLI invocation because that's not
  the common pattern to allow users access docker CLI w/o sudo.
* Fix opt bin path hack for localhost delegate to ignore errors when
  it fails with "sudo password required" otherwise.
* Describe download_run_once with download_localhost use case in docs
  as well.

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-11-24 18:31:26 +01:00
Aleksandr Didenko
db03f17486 Set defaults for ansible_ssh_user
When setting permission for containers download/upload dir we're
using `ansible_ssh_user`. But if playbook is executed without
user being explicitly set `ansible_ssh_user` may be undefined.
In such situations dir ownership will default to `ansible_user_id`

Closes: #644
2016-11-22 18:00:56 +01:00
Bogdan Dobrelya
a03540dabc Add download localhost and enable for CI
* Add download_localhost for the download_run_once mode, which is
  use the ansible host (a travis node for CI case) to store and
  distribute containers across cluster nodes in inventory.
  Defaults to false.
* Rework download_run_once logic to fix idempotency of uploading
  containers.
* For Travis CI, enable docker images caching and run Travis
  workers with sudo enabled as a dependency
* For Travis CI, deploy with download_localhost and download_run_once
  enabled to shourten dev path drastically.
* Add compression for saved container images. Defaults to 'best'.

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
Co-authored-by: Aleksandr Didenko <adidenko@mirantis.com>
2016-11-18 16:00:07 +01:00
Aleksandr Didenko
3e687bbe9a Fix download_run_once for containers
Add one more step (task) to containers download/upload sequence -
copy saved .tar containers to ansible host (delegate_to: localhost).

Then upload images to target nodes. It uses synchronize module so
if ansible host (localhost) is the same host as kube-master[0] then
new task causes no issues and the copy to localhost process is
basically skipped.
2016-11-18 12:47:35 +01:00
Bogdan Dobrelya
c59c3a1bcf Fix idempotency/recurrence of download and preinstall
* Don't push containers if not changed
* Do preinstall role only once and redistribute defaults to
  corresponding roles

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-10-24 18:28:53 +02:00
Smaine Kahlouch
648aa7422d Merge pull request #522 from anthonyhaussman/KubeVersionDefaults
Move kube_version var to defaults
2016-10-05 17:11:59 +02:00
Bogdan Dobrelya
a6a5d0e068 Skip download_run_once for binaries as unimplemented yet
Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-09-30 10:55:02 +02:00
Smaine Kahlouch
5889f7af0e Merge pull request #515 from adidenko/fix-delegate-to
Fix delegate_to expression in download tasks
2016-09-29 10:36:44 +02:00
Anthony Haussmann
34a27b0127 Move kube_version var to defaults
Move the variable kube_version to defaults to have the possibility to overwrite it via group_vars inventory if needed.
2016-09-28 16:15:18 +02:00
Bogdan Dobrelya
ee69ac857e Fix containers download condition
Save/push/load containers if only download.enabled and download.container

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-09-27 13:44:29 +02:00
Aleksandr Didenko
6caf5b0ac3 Fix delegate_to expression in download tasks
"else omit" is causing problems in this expression. Replacing
it with more strict "inventory_hostname" fixes the issue and
handles `download_run_once` as expected.

Closes issue #514
2016-09-27 11:25:24 +02:00
Bogdan Dobrelya
390764c2b4 Add retry_stagger var for failed download/pushes.
* Add the retry_stagger var to tweak push and retry time strategies.
* Add large deployments related docs.

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-09-15 16:43:58 +02:00
Bogdan Dobrelya
9926395e5b Distribute downloaded artifacts
Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-09-15 16:43:56 +02:00
Bogdan Dobrelya
422428908a Download containers and save all
Move version/repo vars to download role.
Add container to download params, which overrides url/source_url,
if enabled.
Fix networking plugins download depending on kube_network_plugin.

Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-09-15 16:43:56 +02:00
Smana
c4beee38f6 include variables from a distinct file 2016-06-29 14:08:14 +02:00
Paul Czarkowski
7de87d958e turn adduser/download roles into meta roles
This should make things a little more composable,
by making these roles meta roles that perform no
actions by default we allow each role to own its own
resources.
2016-05-22 17:25:52 -05:00
Paul Czarkowski
ba615ff94e race condition in download role under vagrant
using a shared folder can cause race conditions for the download
role as it tries to download files on all the nodes to the same
shared path.  This adds a flag to run the tasks in the download
role on just one node.
2016-05-20 17:04:38 -05:00
teuto.net Netzdienste GmbH
457ed11b49 fixed deprecation warnings regarding bare variables 2016-03-30 10:23:43 +02:00
Antoine Legrand
15ce66b2f5 Kubernetes 1.2.0 2016-03-21 16:54:14 +01:00
Antoine Legrand
72807965a8 Upload files to a separate storage 2016-03-04 17:39:02 +01:00
Antoine Legrand
49a7278563 Set perms on unarchive 2016-01-26 12:17:33 +01:00
Greg Althaus
e7d5b7af67 Force owner and permissions for get_url retrieved
files.  get_url doesn't honor owner and mode is spotty.
2016-01-25 13:30:48 -06:00
Antoine Legrand
dd61f685b8 AddUser Role 2016-01-24 11:54:34 +01:00
Smaine Kahlouch
87b42e34e0 create kube-cert group task 2016-01-22 16:51:54 +01:00
Smaine Kahlouch
cb59559835 use command instead of synchronize 2016-01-22 16:37:07 +01:00
ant31
e3cdb3574a Rework download role 2015-12-31 16:12:16 +01:00
Smaine Kahlouch
7006d56ab8 split role download and preinstall 2015-12-31 14:07:02 +01:00
Smaine Kahlouch
dbb6f4934e common role in order to support other linux distribs 2015-12-30 22:26:45 +01:00
Smaine Kahlouch
6f4f170a88 remove useless etcd download, runs into docker containers 2015-12-30 09:50:02 +01:00
ant31
e378f4fb14 Install calico-plugin before running calico 2015-12-28 22:04:39 +01:00
Smaine Kahlouch
3981b73924 download only required kubernetes binaries 2015-12-12 19:37:08 +01:00
Smaine Kahlouch
00c562828f Initial commit 2015-10-03 22:19:50 +02:00