[2.13] Backport CI fix (#7119 )

* [2.14] fix ci (#7021) * fix flake8 errors in Kubespray CI - tox-inventory-builder * fix flake8 errors in Kubespray CI - tox-inventory-builder * Invalidate CRI-O kubic repo's cache Signed-off-by: Victor Morales <v.morales@samsung.com> * add support to configure pkg install retries and use in CI job tf-ovh_ubuntu18-calico (due to it failing often) * Switch Calico and Cilium image repos to Quay.io Co-authored-by: Victor Morales <v.morales@samsung.com> Co-authored-by: Barry Melbourne <9964974+bmelbourne@users.noreply.github.com> Conflicts: roles/download/defaults/main.yml * up vagrant box to fedora/33-cloud-base in cri-o molecule tests (cherry picked from commit 06ec5393d7) * add Google proxy-mirror-cache for docker hub to CI tests (cherry picked from commit d739a6bb2f) * containerd docker hub registry mirror support * containerd docker hub registry mirror support * add docs * fix typo * fix yamllint * fix indent in sample and ansible-playbook param in testcases_run * fix md * mv common vars to tests/common/_docker_hub_registry_mirror.yml * checkout vars to upgrade tests (cherry picked from commit 4a8a52bad9) * Exclude .git/ from shellcheck If a branch name contains '.sh', current shellcheck checks the branch file under .git/ and outputs error because the format is not shell script one. This makes shellcheck exclude files under .git/ to avoid this issue. (cherry picked from commit e2467d87b6) Co-authored-by: Hans Feldt <2808287+hafe@users.noreply.github.com> Co-authored-by: Sergey <s.bondarev@southbridge.ru> Co-authored-by: Kenichi Omichi <ken-oomichi@wx.jp.nec.com> * Switch some image from dockerhub to k8s.gcr (also increase pkg retries) (#6955) * Set packet_centos8-kube-ovn test to manual Signed-off-by: Rick Haan <rickhaan94@gmail.com> Co-authored-by: Etienne Champetier <champetier.etienne@gmail.com> Co-authored-by: Hans Feldt <2808287+hafe@users.noreply.github.com> Co-authored-by: Sergey <s.bondarev@southbridge.ru> Co-authored-by: Kenichi Omichi <ken-oomichi@wx.jp.nec.com> Co-authored-by: Florian Ruynat <16313165+floryut@users.noreply.github.com>
Add Kubernetes hashes 1.18.9/1.17.12 and set default (#6701 )
2021-01-21 06:49:04 -08:00 · 2020-09-18 02:08:45 -07:00 · 2020-09-17 02:46:45 -07:00 · 2020-09-07 01:23:41 -07:00 · 2020-08-04 05:10:19 -07:00 · 2020-08-01 00:29:41 -07:00
45 changed files with 551 additions and 253 deletions
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@ -8,13 +8,14 @@ stages:
  - deploy-special

 variables:
-  KUBESPRAY_VERSION: v2.12.6
+  KUBESPRAY_VERSION: v2.12.9
  FAILFASTCI_NAMESPACE: 'kargo-ci'
  GITLAB_REPOSITORY: 'kargo-ci/kubernetes-sigs-kubespray'
  ANSIBLE_FORCE_COLOR: "true"
  MAGIC: "ci check this"
  TEST_ID: "$CI_PIPELINE_ID-$CI_BUILD_ID"
  CI_TEST_VARS: "./tests/files/${CI_JOB_NAME}.yml"
+  CI_TEST_REGISTRY_MIRROR: "./tests/common/_docker_hub_registry_mirror.yml"
  GS_ACCESS_KEY_ID: $GS_KEY
  GS_SECRET_ACCESS_KEY: $GS_SECRET
  CONTAINER_ENGINE: docker
--- a/.gitlab-ci/packet.yml
+++ b/.gitlab-ci/packet.yml
@ -29,51 +29,90 @@ packet_centos7-flannel-containerd-addons-ha:
  variables:
    MITOGEN_ENABLE: "true"

-packet_ubuntu18-crio:
+packet_centos7-crio:
  extends: .packet
  stage: deploy-part2
  when: on_success
  variables:
    MITOGEN_ENABLE: "true"

-# ### MANUAL JOBS
-
-packet_centos7-weave-upgrade-ha:
-  stage: deploy-part3
+packet_ubuntu18-crio:
  extends: .packet
-  when: on_success
-  variables:
-    UPGRADE_TEST: basic
-    MITOGEN_ENABLE: "false"
-
-packet_ubuntu16-weave-sep:
  stage: deploy-part2
-  extends: .packet
-  when: manual
-
-# # More builds for PRs/merges (manual) and triggers (auto)
-
-packet_ubuntu16-canal-sep:
-  stage: deploy-special
-  extends: .packet
  when: manual
+  variables:
+    MITOGEN_ENABLE: "true"

 packet_ubuntu16-canal-kubeadm-ha:
  stage: deploy-part2
  extends: .packet
  when: on_success

+packet_ubuntu16-canal-sep:
+  stage: deploy-special
+  extends: .packet
+  when: manual
+
 packet_ubuntu16-flannel-ha:
  stage: deploy-part2
  extends: .packet
  when: manual

+packet_ubuntu16-kube-router-sep:
+  stage: deploy-part2
+  extends: .packet
+  when: manual
+
+packet_debian10-containerd:
+  stage: deploy-part2
+  extends: .packet
+  when: on_success
+  variables:
+    MITOGEN_ENABLE: "true"
+
+packet_centos7-calico-ha-once-localhost:
+  stage: deploy-part2
+  extends: .packet
+  when: on_success
+  variables:
+    # This will instruct Docker not to start over TLS.
+    DOCKER_TLS_CERTDIR: ""
+  services:
+    - docker:19.03.9-dind
+
+packet_centos8-kube-ovn:
+  stage: deploy-part2
+  extends: .packet
+  when: manual
+
+packet_centos8-calico:
+  stage: deploy-part2
+  extends: .packet
+  when: on_success
+
+packet_fedora30-weave:
+  stage: deploy-part2
+  extends: .packet
+  when: on_success
+
+packet_opensuse-canal:
+  stage: deploy-part2
+  extends: .packet
+  when: on_success
+
 # Contiv does not work in k8s v1.16
 # packet_ubuntu16-contiv-sep:
 #   stage: deploy-part2
 #   extends: .packet
 #   when: on_success

+# ### MANUAL JOBS
+
+packet_ubuntu16-weave-sep:
+  stage: deploy-part2
+  extends: .packet
+  when: manual
+
 packet_ubuntu18-cilium-sep:
  stage: deploy-special
  extends: .packet
@ -94,50 +133,11 @@ packet_debian9-macvlan:
  extends: .packet
  when: manual

-packet_debian9-calico-upgrade-once:
-  stage: deploy-part3
-  extends: .packet
-  when: on_success
-  variables:
-    UPGRADE_TEST: graceful
-    MITOGEN_ENABLE: "false"
-
-packet_debian10-containerd:
-  stage: deploy-part2
-  extends: .packet
-  when: on_success
-  variables:
-    MITOGEN_ENABLE: "true"
-
 packet_centos7-calico-ha:
  stage: deploy-part2
  extends: .packet
  when: manual

-packet_centos7-calico-ha-once-localhost:
-  stage: deploy-part2
-  extends: .packet
-  when: on_success
-  services:
-    - docker:18.09.9-dind
-
-packet_centos8-kube-ovn:
-  stage: deploy-part2
-  extends: .packet
-  when: on_success
-
-packet_fedora30-weave:
-  stage: deploy-part2
-  extends: .packet
-  when: on_success
-
-packet_fedora31-flannel:
-  stage: deploy-part2
-  extends: .packet
-  when: on_success
-  variables:
-    MITOGEN_ENABLE: "true"
-
 packet_centos7-kube-router:
  stage: deploy-part2
  extends: .packet
@ -148,25 +148,17 @@ packet_centos7-multus-calico:
  extends: .packet
  when: manual

-packet_centos8-calico:
-  stage: deploy-part2
-  extends: .packet
-  when: on_success
-
-packet_opensuse-canal:
-  stage: deploy-part2
-  extends: .packet
-  when: on_success
-
 packet_oracle7-canal-ha:
  stage: deploy-part2
  extends: .packet
  when: manual

-packet_ubuntu16-kube-router-sep:
+packet_fedora31-flannel:
  stage: deploy-part2
  extends: .packet
-  when: manual
+  when: on_success
+  variables:
+    MITOGEN_ENABLE: "true"

 packet_amazon-linux-2-aio:
  stage: deploy-part2
@ -176,10 +168,26 @@ packet_amazon-linux-2-aio:
 # ### PR JOBS PART3
 # Long jobs (45min+)

+packet_centos7-weave-upgrade-ha:
+  stage: deploy-part3
+  extends: .packet
+  when: manual
+  variables:
+    UPGRADE_TEST: basic
+    MITOGEN_ENABLE: "false"
+
 packet_debian9-calico-upgrade:
  stage: deploy-part3
  extends: .packet
-  when: on_success
+  when: manual
+  variables:
+    UPGRADE_TEST: graceful
+    MITOGEN_ENABLE: "false"
+
+packet_debian9-calico-upgrade-once:
+  stage: deploy-part3
+  extends: .packet
+  when: manual
  variables:
    UPGRADE_TEST: graceful
    MITOGEN_ENABLE: "false"
--- a/.gitlab-ci/shellcheck.yml
+++ b/.gitlab-ci/shellcheck.yml
@ -7,10 +7,10 @@ shellcheck:
    SHELLCHECK_VERSION: v0.6.0
  before_script:
    - ./tests/scripts/rebase.sh
-    - curl --silent "https://storage.googleapis.com/shellcheck/shellcheck-"${SHELLCHECK_VERSION}".linux.x86_64.tar.xz" | tar -xJv
+    - curl --silent --location "https://github.com/koalaman/shellcheck/releases/download/"${SHELLCHECK_VERSION}"/shellcheck-"${SHELLCHECK_VERSION}".linux.x86_64.tar.xz" | tar -xJv
    - cp shellcheck-"${SHELLCHECK_VERSION}"/shellcheck /usr/bin/
    - shellcheck --version
  script:
    # Run shellcheck for all *.sh except contrib/
-    - find . -name '*.sh' -not -path './contrib/*' | xargs shellcheck --severity error
+    - find . -name '*.sh' -not -path './contrib/*' -not -path './.git/*' | xargs shellcheck --severity error
  except: ['triggers', 'master']
--- a/.gitlab-ci/terraform.yml
+++ b/.gitlab-ci/terraform.yml
@ -98,80 +98,80 @@ tf-validate-aws:
 #     TF_VAR_public_key_path: ""
 #     TF_VAR_operating_system: ubuntu_18_04

-.ovh_variables: &ovh_variables
-  OS_AUTH_URL: https://auth.cloud.ovh.net/v3
-  OS_PROJECT_ID: 8d3cd5d737d74227ace462dee0b903fe
-  OS_PROJECT_NAME: "9361447987648822"
-  OS_USER_DOMAIN_NAME: Default
-  OS_PROJECT_DOMAIN_ID: default
-  OS_USERNAME: 8XuhBMfkKVrk
-  OS_REGION_NAME: UK1
-  OS_INTERFACE: public
-  OS_IDENTITY_API_VERSION: "3"
+# .ovh_variables: &ovh_variables
+#   OS_AUTH_URL: https://auth.cloud.ovh.net/v3
+#   OS_PROJECT_ID: 8d3cd5d737d74227ace462dee0b903fe
+#   OS_PROJECT_NAME: "9361447987648822"
+#   OS_USER_DOMAIN_NAME: Default
+#   OS_PROJECT_DOMAIN_ID: default
+#   OS_USERNAME: 8XuhBMfkKVrk
+#   OS_REGION_NAME: UK1
+#   OS_INTERFACE: public
+#   OS_IDENTITY_API_VERSION: "3"

-tf-ovh_cleanup:
-  stage: unit-tests
-  tags: [light]
-  image: python
-  variables:
-    <<: *ovh_variables
-  before_script:
-    - pip install -r scripts/openstack-cleanup/requirements.txt
-  script:
-    - ./scripts/openstack-cleanup/main.py
+# tf-ovh_cleanup:
+#   stage: unit-tests
+#   tags: [light]
+#   image: python
+#   variables:
+#     <<: *ovh_variables
+#   before_script:
+#     - pip install -r scripts/openstack-cleanup/requirements.txt
+#   script:
+#     - ./scripts/openstack-cleanup/main.py

-tf-ovh_ubuntu18-calico:
-  extends: .terraform_apply
-  when: on_success
-  variables:
-    <<: *ovh_variables
-    TF_VERSION: 0.12.24
-    PROVIDER: openstack
-    CLUSTER: $CI_COMMIT_REF_NAME
-    ANSIBLE_TIMEOUT: "60"
-    SSH_USER: ubuntu
-    TF_VAR_number_of_k8s_masters: "0"
-    TF_VAR_number_of_k8s_masters_no_floating_ip: "1"
-    TF_VAR_number_of_k8s_masters_no_floating_ip_no_etcd: "0"
-    TF_VAR_number_of_etcd: "0"
-    TF_VAR_number_of_k8s_nodes: "0"
-    TF_VAR_number_of_k8s_nodes_no_floating_ip: "1"
-    TF_VAR_number_of_gfs_nodes_no_floating_ip: "0"
-    TF_VAR_number_of_bastions: "0"
-    TF_VAR_number_of_k8s_masters_no_etcd: "0"
-    TF_VAR_use_neutron: "0"
-    TF_VAR_floatingip_pool: "Ext-Net"
-    TF_VAR_external_net: "6011fbc9-4cbf-46a4-8452-6890a340b60b"
-    TF_VAR_network_name: "Ext-Net"
-    TF_VAR_flavor_k8s_master: "defa64c3-bd46-43b4-858a-d93bbae0a229"    # s1-8
-    TF_VAR_flavor_k8s_node: "defa64c3-bd46-43b4-858a-d93bbae0a229"      # s1-8
-    TF_VAR_image: "Ubuntu 18.04"
-    TF_VAR_k8s_allowed_remote_ips: '["0.0.0.0/0"]'
+# tf-ovh_ubuntu18-calico:
+#   extends: .terraform_apply
+#   when: on_success
+#   variables:
+#     <<: *ovh_variables
+#     TF_VERSION: 0.12.24
+#     PROVIDER: openstack
+#     CLUSTER: $CI_COMMIT_REF_NAME
+#     ANSIBLE_TIMEOUT: "60"
+#     SSH_USER: ubuntu
+#     TF_VAR_number_of_k8s_masters: "0"
+#     TF_VAR_number_of_k8s_masters_no_floating_ip: "1"
+#     TF_VAR_number_of_k8s_masters_no_floating_ip_no_etcd: "0"
+#     TF_VAR_number_of_etcd: "0"
+#     TF_VAR_number_of_k8s_nodes: "0"
+#     TF_VAR_number_of_k8s_nodes_no_floating_ip: "1"
+#     TF_VAR_number_of_gfs_nodes_no_floating_ip: "0"
+#     TF_VAR_number_of_bastions: "0"
+#     TF_VAR_number_of_k8s_masters_no_etcd: "0"
+#     TF_VAR_use_neutron: "0"
+#     TF_VAR_floatingip_pool: "Ext-Net"
+#     TF_VAR_external_net: "6011fbc9-4cbf-46a4-8452-6890a340b60b"
+#     TF_VAR_network_name: "Ext-Net"
+#     TF_VAR_flavor_k8s_master: "defa64c3-bd46-43b4-858a-d93bbae0a229"    # s1-8
+#     TF_VAR_flavor_k8s_node: "defa64c3-bd46-43b4-858a-d93bbae0a229"      # s1-8
+#     TF_VAR_image: "Ubuntu 18.04"
+#     TF_VAR_k8s_allowed_remote_ips: '["0.0.0.0/0"]'

-tf-ovh_coreos-calico:
-  extends: .terraform_apply
-  when: on_success
-  variables:
-    <<: *ovh_variables
-    TF_VERSION: 0.12.24
-    PROVIDER: openstack
-    CLUSTER: $CI_COMMIT_REF_NAME
-    ANSIBLE_TIMEOUT: "60"
-    SSH_USER: core
-    TF_VAR_number_of_k8s_masters: "0"
-    TF_VAR_number_of_k8s_masters_no_floating_ip: "1"
-    TF_VAR_number_of_k8s_masters_no_floating_ip_no_etcd: "0"
-    TF_VAR_number_of_etcd: "0"
-    TF_VAR_number_of_k8s_nodes: "0"
-    TF_VAR_number_of_k8s_nodes_no_floating_ip: "1"
-    TF_VAR_number_of_gfs_nodes_no_floating_ip: "0"
-    TF_VAR_number_of_bastions: "0"
-    TF_VAR_number_of_k8s_masters_no_etcd: "0"
-    TF_VAR_use_neutron: "0"
-    TF_VAR_floatingip_pool: "Ext-Net"
-    TF_VAR_external_net: "6011fbc9-4cbf-46a4-8452-6890a340b60b"
-    TF_VAR_network_name: "Ext-Net"
-    TF_VAR_flavor_k8s_master: "4d4fd037-9493-4f2b-9afe-b542b5248eac"    # b2-7
-    TF_VAR_flavor_k8s_node: "4d4fd037-9493-4f2b-9afe-b542b5248eac"      # b2-7
-    TF_VAR_image: "CoreOS Stable"
-    TF_VAR_k8s_allowed_remote_ips: '["0.0.0.0/0"]'
+# tf-ovh_coreos-calico:
+#   extends: .terraform_apply
+#   when: on_success
+#   variables:
+#     <<: *ovh_variables
+#     TF_VERSION: 0.12.24
+#     PROVIDER: openstack
+#     CLUSTER: $CI_COMMIT_REF_NAME
+#     ANSIBLE_TIMEOUT: "60"
+#     SSH_USER: core
+#     TF_VAR_number_of_k8s_masters: "0"
+#     TF_VAR_number_of_k8s_masters_no_floating_ip: "1"
+#     TF_VAR_number_of_k8s_masters_no_floating_ip_no_etcd: "0"
+#     TF_VAR_number_of_etcd: "0"
+#     TF_VAR_number_of_k8s_nodes: "0"
+#     TF_VAR_number_of_k8s_nodes_no_floating_ip: "1"
+#     TF_VAR_number_of_gfs_nodes_no_floating_ip: "0"
+#     TF_VAR_number_of_bastions: "0"
+#     TF_VAR_number_of_k8s_masters_no_etcd: "0"
+#     TF_VAR_use_neutron: "0"
+#     TF_VAR_floatingip_pool: "Ext-Net"
+#     TF_VAR_external_net: "6011fbc9-4cbf-46a4-8452-6890a340b60b"
+#     TF_VAR_network_name: "Ext-Net"
+#     TF_VAR_flavor_k8s_master: "4d4fd037-9493-4f2b-9afe-b542b5248eac"    # b2-7
+#     TF_VAR_flavor_k8s_node: "4d4fd037-9493-4f2b-9afe-b542b5248eac"      # b2-7
+#     TF_VAR_image: "CoreOS Stable"
+#     TF_VAR_k8s_allowed_remote_ips: '["0.0.0.0/0"]'
--- a/README.md
+++ b/README.md
@ -114,13 +114,13 @@ Note: Upstart/SysV init based OS types are not supported.
 ## Supported Components

 - Core
-  - [kubernetes](https://github.com/kubernetes/kubernetes) v1.17.5
+  - [kubernetes](https://github.com/kubernetes/kubernetes) v1.17.12
  - [etcd](https://github.com/coreos/etcd) v3.3.12
  - [docker](https://www.docker.com/) v18.06 (see note)
  - [containerd](https://containerd.io/) v1.2.13
  - [cri-o](http://cri-o.io/) v1.17 (experimental: see [CRI-O Note](docs/cri-o.md). Only on fedora, ubuntu and centos based OS)
 - Network Plugin
-  - [cni-plugins](https://github.com/containernetworking/plugins) v0.8.5
+  - [cni-plugins](https://github.com/containernetworking/plugins) v0.8.6
  - [calico](https://github.com/projectcalico/calico) v3.13.2
  - [canal](https://github.com/projectcalico/canal) (given calico/flannel versions)
  - [cilium](https://github.com/cilium/cilium) v1.7.2
--- a/contrib/inventory_builder/tests/test_inventory.py
+++ b/contrib/inventory_builder/tests/test_inventory.py
@ -51,7 +51,7 @@ class TestInventory(unittest.TestCase):
        groups = ['group1', 'group2']
        self.inv.ensure_required_groups(groups)
        for group in groups:
-            self.assertTrue(group in self.inv.yaml_config['all']['children'])
+            self.assertIn(group, self.inv.yaml_config['all']['children'])

    def test_get_host_id(self):
        hostnames = ['node99', 'no99de01', '01node01', 'node1.domain',
@ -209,8 +209,8 @@ class TestInventory(unittest.TestCase):
            ('doesnotbelong2', {'whateveropts=ilike'})])
        self.inv.yaml_config['all']['hosts'] = existing_hosts
        self.inv.purge_invalid_hosts(proper_hostnames)
-        self.assertTrue(
-            bad_host not in self.inv.yaml_config['all']['hosts'].keys())
+        self.assertNotIn(
+            bad_host, self.inv.yaml_config['all']['hosts'].keys())

    def test_add_host_to_group(self):
        group = 'etcd'
@ -227,8 +227,8 @@ class TestInventory(unittest.TestCase):
        host = 'node1'

        self.inv.set_kube_master([host])
-        self.assertTrue(
-            host in self.inv.yaml_config['all']['children'][group]['hosts'])
+        self.assertIn(
+            host, self.inv.yaml_config['all']['children'][group]['hosts'])

    def test_set_all(self):
        hosts = OrderedDict([
@ -246,8 +246,8 @@ class TestInventory(unittest.TestCase):

        self.inv.set_k8s_cluster()
        for host in expected_hosts:
-            self.assertTrue(
-                host in
+            self.assertIn(
+                host,
                self.inv.yaml_config['all']['children'][group]['children'])

    def test_set_kube_node(self):
@ -255,16 +255,16 @@ class TestInventory(unittest.TestCase):
        host = 'node1'

        self.inv.set_kube_node([host])
-        self.assertTrue(
-            host in self.inv.yaml_config['all']['children'][group]['hosts'])
+        self.assertIn(
+            host, self.inv.yaml_config['all']['children'][group]['hosts'])

    def test_set_etcd(self):
        group = 'etcd'
        host = 'node1'

        self.inv.set_etcd([host])
-        self.assertTrue(
-            host in self.inv.yaml_config['all']['children'][group]['hosts'])
+        self.assertIn(
+            host, self.inv.yaml_config['all']['children'][group]['hosts'])

    def test_scale_scenario_one(self):
        num_nodes = 50
--- a/docs/ci.md
+++ b/docs/ci.md
@ -9,7 +9,6 @@ To generate this Matrix run `./tests/scripts/md-table/main.py`
 amazon |  :white_check_mark: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 centos7 |  :white_check_mark: | :x: | :x: | :x: | :x: | :x: | :white_check_mark: | :x: | :white_check_mark: |
 centos8 |  :white_check_mark: | :x: | :x: | :x: | :x: | :white_check_mark: | :x: | :x: | :x: |
-coreos |  :white_check_mark: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 debian10 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 debian9 |  :white_check_mark: | :x: | :x: | :x: | :x: | :x: | :x: | :white_check_mark: | :x: |
 fedora30 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :white_check_mark: |
@ -25,9 +24,8 @@ ubuntu20 |  :white_check_mark: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 | OS / CNI | calico | canal | cilium | contiv | flannel | kube-ovn | kube-router | macvlan | weave |
 |---| --- | --- | --- | --- | --- | --- | --- | --- | --- |
 amazon |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
-centos7 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
+centos7 |  :white_check_mark: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 centos8 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
-coreos |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 debian10 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 debian9 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 fedora30 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
@ -45,7 +43,6 @@ ubuntu20 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 amazon |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 centos7 |  :x: | :x: | :x: | :x: | :white_check_mark: | :x: | :x: | :x: | :x: |
 centos8 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
-coreos |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 debian10 |  :white_check_mark: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 debian9 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
 fedora30 |  :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: |
--- a/docs/containerd.md
+++ b/docs/containerd.md
@ -0,0 +1,31 @@
+# conrainerd
+
+[containerd] An industry-standard container runtime with an emphasis on simplicity, robustness and portability
+Kubespray supports basic functionality for using containerd as the default container runtime in a cluster.
+
+_To use the containerd container runtime set the following variables:_
+
+## k8s-cluster.yml
+
+```yaml
+container_manager: containerd
+```
+
+## Containerd config
+
+Example: define registry mirror for docker hub
+
+```yaml
+containerd_config:
+  grpc:
+    max_recv_message_size: 16777216
+    max_send_message_size: 16777216
+  debug:
+    level: ""
+  registries:
+    "docker.io":
+      - "https://mirror.gcr.io"
+      - "https://registry-1.docker.io"
+```
+
+[containerd]: https://containerd.io/
--- a/docs/integration.md
+++ b/docs/integration.md
@ -7,7 +7,7 @@

 2. Add **forked repo** as submodule to desired folder in your existent ansible repo(for example 3d/kubespray):
  ```git submodule add https://github.com/YOUR_GITHUB/kubespray.git kubespray```
-  Git will create _.gitmodules_ file in your existent ansible repo:
+  Git will create `.gitmodules` file in your existent ansible repo:

   ```ini
   [submodule "3d/kubespray"]
--- a/inventory/sample/group_vars/all/containerd.yml
+++ b/inventory/sample/group_vars/all/containerd.yml
@ -1,6 +1,8 @@
 ---
 # Please see roles/container-engine/containerd/defaults/main.yml for more configuration options

+# Example: define registry mirror for docker hub
+
 # containerd_config:
 #   grpc:
 #     max_recv_message_size: 16777216
@ -8,7 +10,9 @@
 #   debug:
 #     level: ""
 #   registries:
-#     "docker.io": "https://registry-1.docker.io"
+#     "docker.io":
+#       - "https://mirror.gcr.io"
+#       - "https://registry-1.docker.io"
 #   max_container_log_line_size: -1
 #   metrics:
 #     address: ""
--- a/inventory/sample/group_vars/k8s-cluster/addons.yml
+++ b/inventory/sample/group_vars/k8s-cluster/addons.yml
@ -1,7 +1,7 @@
 ---
 # Kubernetes dashboard
 # RBAC required. see docs/getting-started.md for access details.
-dashboard_enabled: true
+# dashboard_enabled: true

 # Helm deployment
 helm_enabled: false
--- a/inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml
+++ b/inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml
@ -20,7 +20,7 @@ kube_users_dir: "{{ kube_config_dir }}/users"
 kube_api_anonymous_auth: true

 ## Change this to use another Kubernetes version, e.g. a current beta release
-kube_version: v1.17.5
+kube_version: v1.17.12

 # kubernetes image repo define
 kube_image_repo: "k8s.gcr.io"
--- a/remove-node.yml
+++ b/remove-node.yml
@ -2,6 +2,21 @@
 - name: Check ansible version
  import_playbook: ansible_version.yml

+- hosts: all
+  gather_facts: false
+  tags: always
+  tasks:
+    - name: "Set up proxy environment"
+      set_fact:
+        proxy_env:
+          http_proxy: "{{ http_proxy | default ('') }}"
+          HTTP_PROXY: "{{ http_proxy | default ('') }}"
+          https_proxy: "{{ https_proxy | default ('') }}"
+          HTTPS_PROXY: "{{ https_proxy | default ('') }}"
+          no_proxy: "{{ no_proxy | default ('') }}"
+          NO_PROXY: "{{ no_proxy | default ('') }}"
+      no_log: true
+
 - hosts: "{{ node | default('etcd:k8s-cluster:calico-rr') }}"
  gather_facts: no
  vars_prompt:
--- a/reset.yml
+++ b/reset.yml
@ -2,6 +2,21 @@
 - name: Check ansible version
  import_playbook: ansible_version.yml

+- hosts: all
+  gather_facts: false
+  tags: always
+  tasks:
+    - name: "Set up proxy environment"
+      set_fact:
+        proxy_env:
+          http_proxy: "{{ http_proxy | default ('') }}"
+          HTTP_PROXY: "{{ http_proxy | default ('') }}"
+          https_proxy: "{{ https_proxy | default ('') }}"
+          HTTPS_PROXY: "{{ https_proxy | default ('') }}"
+          no_proxy: "{{ no_proxy | default ('') }}"
+          NO_PROXY: "{{ no_proxy | default ('') }}"
+      no_log: true
+
 - hosts: bastion[0]
  gather_facts: False
  roles:
--- a/roles/container-engine/containerd/templates/config.toml.j2
+++ b/roles/container-engine/containerd/templates/config.toml.j2
@ -55,7 +55,7 @@ disabled_plugins = ["restart"]
 [plugins.cri.registry.mirrors]
 {% for registry, addr in containerd_config.registries.items() %}
 [plugins.cri.registry.mirrors."{{ registry }}"]
-  endpoint = ["{{ addr }}"]
+  endpoint = ["{{ ([ addr ] | flatten ) | join('","') }}"]
 {% endfor %}
 {% endif %}

--- a/roles/container-engine/cri-o/defaults/main.yml
+++ b/roles/container-engine/cri-o/defaults/main.yml
@ -9,7 +9,7 @@ crio_conmon: "/usr/bin/conmon"
 crio_required_version: "{{ kube_version | regex_replace('^v(?P<major>\\d+).(?P<minor>\\d+).(?P<patch>\\d+)$', '\\g<major>.\\g<minor>') }}"

 crio_kubernetes_version_matrix:
-  "1.18": "1.17"
+  "1.18": "1.18"
  "1.17": "1.17"
  "1.16": "1.16"

--- a/roles/container-engine/cri-o/molecule/default/molecule.yml
+++ b/roles/container-engine/cri-o/molecule/default/molecule.yml
@ -27,7 +27,7 @@ platforms:
    groups:
      - kube-master
  - name: fedora
-    box: fedora/31-cloud-base
+    box: fedora/33-cloud-base
    cpus: 2
    memory: 1024
    groups:
--- a/roles/container-engine/cri-o/tasks/crio_repo.yml
+++ b/roles/container-engine/cri-o/tasks/crio_repo.yml
@ -22,18 +22,19 @@
  yum_repository:
    name: devel_kubic_libcontainers_stable
    description: Stable Releases of Upstream github.com/containers packages (CentOS_$releasever)
-    baseurl: http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_$releasever/
+    baseurl: http://widehat.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_$releasever/
    gpgcheck: yes
    gpgkey: http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_$releasever/repodata/repomd.xml.key
+    keepcache: false
  when: ansible_distribution in ["CentOS"]

 - name: Add CRI-O kubic repo
  yum_repository:
    name: "devel_kubic_libcontainers_stable_cri-o_{{ crio_version }}"
-    description: 1.17 (CentOS_$releasever)
-    baseurl: "http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/{{ crio_version }}/CentOS_$releasever/"
+    description: "CRI-O {{ crio_version }} (CentOS_$releasever)"
+    baseurl: "http://widehat.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/{{ crio_version }}/CentOS_$releasever/"
    gpgcheck: yes
-    gpgkey: "http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/{{ crio_version }}/CentOS_$releasever/repodata/repomd.xml.key"
+    gpgkey: "http://widehat.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/{{ crio_version }}/CentOS_$releasever/repodata/repomd.xml.key"
  when: ansible_distribution in ["CentOS"]

 - name: Enable modular repos for CRI-O
--- a/roles/container-engine/cri-o/templates/crio.conf.j2
+++ b/roles/container-engine/cri-o/templates/crio.conf.j2
@ -106,7 +106,7 @@ seccomp_profile = "{{crio_seccomp_profile}}"

 # Used to change the name of the default AppArmor profile of CRI-O. The default
 # profile name is "crio-default-" followed by the version string of CRI-O.
-apparmor_profile = "crio-default"
+# apparmor_profile = "crio-default"

 # Cgroup management implementation used for the runtime.
 cgroup_manager = "{{crio_cgroup_manager}}"
--- a/roles/container-engine/cri-o/vars/ubuntu.yml
+++ b/roles/container-engine/cri-o/vars/ubuntu.yml
@ -4,3 +4,8 @@ crio_packages:
  - "cri-o-{{ crio_version }}"

 crio_runc_path: /usr/sbin/runc
+
+crio_kubernetes_version_matrix:
+  "1.18": "1.17"
+  "1.17": "1.17"
+  "1.16": "1.16"
--- a/roles/container-engine/docker/defaults/main.yml
+++ b/roles/container-engine/docker/defaults/main.yml
@ -1,5 +1,6 @@
 ---
 docker_version: '18.09'
+docker_cli_version: "{{ 'latest' if docker_version != 'latest' and docker_version is version('18.09', '<') else docker_version }}"
 docker_selinux_version: '17.03'

 docker_package_info:
--- a/roles/container-engine/docker/vars/debian.yml
+++ b/roles/container-engine/docker/vars/debian.yml
@ -16,9 +16,16 @@ docker_versioned_pkg:
  'stable': docker-ce=5:18.09.7~3-0~debian-{{ ansible_distribution_release|lower }}
  'edge': docker-ce=5:19.03.7~3-0~debian-{{ ansible_distribution_release|lower }}

+docker_cli_versioned_pkg:
+  'latest': docker-ce-cli
+  '18.09': docker-ce-cli=5:18.09.9~3-0~debian-{{ ansible_distribution_release|lower }}
+  '19.03': docker-ce-cli=5:19.03.9~3-0~debian-{{ ansible_distribution_release|lower }}
+
 docker_package_info:
  pkg_mgr: apt
  pkgs:
+    - name: "{{ docker_cli_versioned_pkg[docker_cli_version | string] }}"
+      force: yes
    - name: "{{ docker_versioned_pkg[docker_version | string] }}"
      force: yes

--- a/roles/container-engine/docker/vars/fedora.yml
+++ b/roles/container-engine/docker/vars/fedora.yml
@ -2,21 +2,26 @@
 docker_kernel_min_version: '0'

 # https://docs.docker.com/install/linux/docker-ce/fedora/
-# https://download.docker.com/linux/fedora/28/x86_64/stable/Packages/
+# https://download.docker.com/linux/fedora/<fedora-version>/x86_64/stable/Packages/

 docker_versioned_pkg:
  'latest': docker-ce
-  '18.03': docker-ce-18.03.1.ce-3.fc28
-  '18.06': docker-ce-18.06.2.ce-3.fc28
-  '18.09': docker-ce-18.09.7-3.fc28
+  '18.03': docker-ce-18.03.1.ce-3.fc{{ ansible_distribution_major_version }}
+  '18.06': docker-ce-18.06.2.ce-3.fc{{ ansible_distribution_major_version }}
+  '18.09': docker-ce-18.09.7-3.fc{{ ansible_distribution_major_version }}
+  '19.03': docker-ce-19.03.8-3.fc{{ ansible_distribution_major_version }}

-#
-# This is due to the fact that the docker
-# packages available on Fedora are too recent
-#
-docker_version: "latest"
+docker_cli_versioned_pkg:
+  'latest': docker-ce-cli
+  '18.09': docker-ce-cli-19.03.8-3.fc{{ ansible_distribution_major_version }}
+  '19.03': docker-ce-cli-19.03.9-3.fc{{ ansible_distribution_major_version }}
+
+# Fedora 30/31 don't provide packages for docker 18.0x ...
+docker_version: "19.03"
+docker_cli_version: "19.03"

 docker_package_info:
  pkg_mgr: dnf
  pkgs:
+    - name: "{{ docker_cli_versioned_pkg[docker_cli_version | string] }}"
    - name: "{{ docker_versioned_pkg[docker_version | string] }}"
--- a/roles/container-engine/docker/vars/redhat.yml
+++ b/roles/container-engine/docker/vars/redhat.yml
@ -16,6 +16,11 @@ docker_versioned_pkg:
  'stable': docker-ce-18.09.9-3.el7
  'edge': docker-ce-19.03.8-3.el7

+docker_cli_versioned_pkg:
+  'latest': docker-ce-cli
+  '18.09': docker-ce-cli-18.09.9-3.el7
+  '19.03': docker-ce-cli-19.03.9-3.el7
+
 docker_selinux_versioned_pkg:
  'latest': docker-ce-selinux-17.03.3.ce-1.el7
  '17.03': docker-ce-selinux-17.03.3.ce-1.el7
@ -30,13 +35,15 @@ docker_pkgs_use_docker_ce:
    yum_conf: "{{ docker_yum_conf }}"

 docker_pkgs:
+  - name: "{{ docker_cli_versioned_pkg[docker_cli_version | string] }}"
+    yum_conf: "{{ docker_yum_conf }}"
  - name: "{{ docker_versioned_pkg[docker_version | string] }}"
    yum_conf: "{{ docker_yum_conf }}"

 docker_package_info:
  pkg_mgr: yum
  pkgs: |-
-    {%- if docker_version is version('17.04', '<') -%}
+    {%- if docker_version != "latest" and docker_version is version('17.04', '<') -%}
    {{ docker_pkgs_use_docker_ce }}
    {%- else -%}
    {{ docker_pkgs }}
--- a/roles/container-engine/docker/vars/ubuntu-amd64.yml
+++ b/roles/container-engine/docker/vars/ubuntu-amd64.yml
@ -16,9 +16,16 @@ docker_versioned_pkg:
  'stable': docker-ce=5:18.09.7~3-0~ubuntu-{{ ansible_distribution_release|lower }}
  'edge': docker-ce=5:19.03.7~3-0~ubuntu-{{ ansible_distribution_release|lower }}

+docker_cli_versioned_pkg:
+  'latest': docker-ce-cli
+  '18.09': docker-ce-cli=5:18.09.9~3-0~ubuntu-{{ ansible_distribution_release|lower }}
+  '19.03': docker-ce-cli=5:19.03.9~3-0~ubuntu-{{ ansible_distribution_release|lower }}
+
 docker_package_info:
  pkg_mgr: apt
  pkgs:
+    - name: "{{ docker_cli_versioned_pkg[docker_cli_version | string] }}"
+      force: yes
    - name: "{{ docker_versioned_pkg[docker_version | string] }}"
      force: yes

--- a/roles/container-engine/docker/vars/ubuntu-arm64.yml
+++ b/roles/container-engine/docker/vars/ubuntu-arm64.yml
@ -12,9 +12,16 @@ docker_versioned_pkg:
  'stable': docker-ce=5:18.09.7~3-0~ubuntu-{{ ansible_distribution_release|lower }}
  'edge': docker-ce=5:19.03.7~3-0~ubuntu-{{ ansible_distribution_release|lower }}

+docker_cli_versioned_pkg:
+  'latest': docker-ce-cli
+  '18.09': docker-ce-cli=5:18.09.9~3-0~ubuntu-{{ ansible_distribution_release|lower }}
+  '19.03': docker-ce-cli=5:19.03.9~3-0~ubuntu-{{ ansible_distribution_release|lower }}
+
 docker_package_info:
  pkg_mgr: apt
  pkgs:
+    - name: "{{ docker_cli_versioned_pkg[docker_cli_version | string] }}"
+      force: yes
    - name: "{{ docker_versioned_pkg[docker_version | string] }}"
      force: yes

--- a/roles/download/defaults/main.yml
+++ b/roles/download/defaults/main.yml
@ -49,7 +49,7 @@ download_delegate: "{% if download_localhost %}localhost{% else %}{{ groups['kub
 image_arch: "{{host_architecture | default('amd64')}}"

 # Versions
-kube_version: v1.17.5
+kube_version: v1.17.12
 kubeadm_version: "{{ kube_version }}"
 etcd_version: v3.3.12

@ -77,7 +77,7 @@ typha_enabled: false

 flannel_version: "v0.12.0"

-cni_version: "v0.8.5"
+cni_version: "v0.8.6"

 weave_version: 2.6.2
 pod_infra_version: 3.1
@ -125,15 +125,33 @@ crictl_checksums:
 # Checksums
 kubelet_checksums:
  arm:
+    v1.18.9: 24f7559fe5214b5f3b625ab035e29159441e6cfd248befbeb78b63e660fccd23
+    v1.18.8: 831f50ea45384884c50395c288d493e75dd134a962dc95261ce122de5e6a17ec
+    v1.18.6: 5f8367f9f5de77c022ec5d0cd86e897b7a33318185eaadb7736033d8dabcdbdb
+    v1.18.5: 9f8ab727964c6f42f1c17089bf2f7b4b2f2a5c61ffab3bad16eb02d9feb05855
+    v1.18.4: 796defe5f8b43a5316a487a377b4059df12b9b3c933f3fe4dff40e8144a11af6
+    v1.18.3: 491344027cbec40bc867a79c7130c27c143648544b5dfe4a28929cf26427dc3b
    v1.18.2: b7b9c43851dde9cbaa2061828410c60ee63e53fbf3ebc5559b7f4387dae67bb9
    v1.18.1: 04d8e0a080dcb23d579c69e769e75bd5abaa1977d43550ec891560d76f1f7f37
    v1.18.0: 985c1a1b492ccc6e46e1cd454790dae539d5b93208efb05e35114f66a183de99
+    v1.17.12: 2b6160f5c15b1294573e37031ce0bcbbcd3ee3ea055f4f8fe4faf3ab74774f40
+    v1.17.11: 53784f9bea1508b6d82f8be6f40af2e6aef86381021b500c1647f9d297af3efd
+    v1.17.9: 37fc89360f4dcff4788032e60fe6388da4194a68503992868261ba840413e8d4
+    v1.17.8: 82320569bc9deff33d148c759a105f1a32de3d83855165100261a4ad395d1845
+    v1.17.7: 3b368039523357959e451a35867b5659701e135ca2069cb9487c7459084c46d9
+    v1.17.6: e522cda9b86de29da72fd306968e1ba44cb85b61a743083f8fee39899a755210
    v1.17.5: d1eb5b7a3a88030490f1619f2e7d723926214ba941e2172112bccb71f41d9aab
    v1.17.4: c8c4d1b869c72b4203024615cafae1cca7df2fb89dd7f4a524d05ffa5edde559
    v1.17.3: 06fe53b9780e4fa17b5e14f588bbaaa09fc0924ef4040e26a484fa3235c9e110
    v1.17.2: 9a2ab021f8556fabcb00022052810b3d8136704141891439de1340ac9e439d6d
    v1.17.1: 0219c940bad3238dfbdf8e4518241d861bbdd8fc93d172cc632c225d7dd57094
    v1.17.0: 75ae6ad8f4a7f2ac3988b37a01c28093f240745d17c1781135d1844057c8ae94
+    v1.16.15: fe4035e1a0a4eb36468cf765e39bb00c7b99bab81e3ba428c77c0296ec6e1039
+    v1.16.14: 17a2051b95aacf409e08325ae7eed6e0a41cf79a9488dba5fa701fa9254e0b71
+    v1.16.13: 0576575832a1e26ff823b4f8ca25cd51e211c3ab43b8880b2a74693c0733cd60
+    v1.16.12: edc35704864fdf5ea28b7bd17580e2155c8599f8f93ed0fb2979f89f747a3f97
+    v1.16.11: adb8c2ecc937b6486d73551121d986924a7e1f503a70d973cc683f8a6ae4f9ab
+    v1.16.10: a1e3b5af8d6e97fdd11154435266f606361978e9a9d0b836f69427c274153c0c
    v1.16.9: 5c08b7754d0230dcd5493ab09e00c2e2397ce795cb450c0807220faa69e87548
    v1.16.8: e3ac093071ad59f726ae0f88d1d10ada7ec559ed2b7817495339f4380ae32057
    v1.16.7: ace84b05e4de55042ed4288eab14f33a5a23519ee7d9981be25b0cf3e74172c2
@ -156,15 +174,33 @@ kubelet_checksums:
    v1.15.1: cb97d18c61b39f0721201fedcb0219b132e521d51a2595c1f7715700626097c6
    v1.15.0: 663c59fca7247ca325d4328cfc3ce77d1d4eddb251137f91ec62b8cd8823d1fc
  arm64:
+    v1.18.9: 21b0fb4682deea19be3ac160403db9858dc9d02b101d60eb6fc22a86523ec434
+    v1.18.8: d36e2d656bad232e8b48b19c948164ee3966669f4566cf5ea43ca22f6eed1aa5
+    v1.18.6: 257fd42be375025fb93724bda9bef23b73eb40531f22bab9e19f6d6ff1ca57cf
+    v1.18.5: c3815bc740755aa9fd3ec240ad808a13628a4deb6ec2b4338e772fd0cf77e1a2
+    v1.18.4: ec4e18e7a2e94fb1ca83d739eadb8d81748cf6a48b87b8fe0d66131e9515e8c6
+    v1.18.3: f88deee2052b4d1e3a15fd7352b93728c23d69497a4199a56e62fa871bdf7edb
    v1.18.2: 89b5066ae17df8488c76a83c70cbcac0771fa36803e31b826f2770b5efcdbfbf
    v1.18.1: 2181cde9e6b24055d262b78758b365363273896968df673eb13d4f17a4f69c4a
    v1.18.0: db91a26f8baa2bce017172305e717e77be5cfc4272592be8cb0155e1cfa7719e
+    v1.17.12: bfd5ff97040a2f1017bc14991757ba62ad8c8218d75a5f646690c1e76f93bf22
+    v1.17.11: 0d9705c284054b2fdb7627d4867edd0863f67a2fbf64a2e1710d928936539c89
+    v1.17.9: d57c25a3d67c937a9d6778de07295478185f73938937868525030a01d15c372f
+    v1.17.8: 673355f62aa422915682ae595e4e53813e4656f2c272eb032f97492211cfced5
+    v1.17.7: eb1715a745281f6aee34644653f73787acdd9f3904e3d58e1319ded4a16be013
+    v1.17.6: 6ded412f13e5d8bd0368372150334580a05cd4dc7629f437c789a5aa6008e8e5
    v1.17.5: 9220a7390d9c5cb5c770d947babdec288d044126b9982bbd5d5c8785354a6701
    v1.17.4: 77ca08cd3d03edda8d628e39a8cb45afe794582a9619d381ec5a70585999721a
    v1.17.3: eeefd2f966dfb75ab4ab58829118f9bb314b75799a94d21c2ce8d083cc330dbc
    v1.17.2: 133b69346da8e34daaf20f421657625a06630ec1e11f06961523836383cea72c
    v1.17.1: c773512ade5da3188ed4c312d5ba01bfbf3f376f6e580e5b074827a5b25450aa
    v1.17.0: b1a4a2325383854a69ec768e7dc00f69378d3ccbc554859d910bf5b582264ea2
+    v1.16.15: 3d8337e2e0b613688d3828803988adeafedc9451c785478951002bf627beb36b
+    v1.16.14: cb4558510d00f3b06f4bf22ec7d7ed8c6417662e6344048ab36d9d3e60e3c028
+    v1.16.13: 66579a5e4f4133d2c66f506311884f7bcc8a0d719f63d119eddc0d4809f97781
+    v1.16.12: 0ef9d42e27bf85e9ff276f2181e17e2912941c3a7ae9086de722ac3c9cea997f
+    v1.16.11: 074a81dbe658bc47b9e5b9a4733743239c40bc83472b745c7c85774bf33ff3ab
+    v1.16.10: 0634e04a13393dfe8604a7798a11f4c3d2cdc9443e26cbc6d3af35e9ac9727ee
    v1.16.9: 10c5dc66f309184389ffb7c2d8d9d4d8f291a81559385b5537bce8f0b8c7e918
    v1.16.8: a6889c9957d8ec3ba15676b1e2eff021c9d120284f185d367626763dd15a245b
    v1.16.7: bdfa1638e285e4665d9888770eb69aabfa6c08bcc8c6a2285bc06909c68fa4a0
@ -187,15 +223,33 @@ kubelet_checksums:
    v1.15.1: 3032531dc8ff4ea1debafcfb7b84f6c39a83e67a452a50b64d1023ba57299100
    v1.15.0: a16443434eb0391991bff9a10288d83beb38be4d406954858d9fdc1063870284
  amd64:
+    v1.18.9: 5028b6548e8838e1e0851f10e8bd8d9a6ef1693e3f1dac09f7d50c4c2873f20b
+    v1.18.8: a4116675ac52bf80e224fba8ff6db6f2d7aed192bf6fffd5f8e4d5efb4368f31
+    v1.18.6: 2eb9baf5a65a7b94c653dbd7af03a768a520961eb27ef369e43ef12711e22d4a
+    v1.18.5: 8c328f65d30f0edd0fd4f529b09d6fc588cfb7b524d5c9f181e36de6e494e19c
+    v1.18.4: 42bcd6a8fe1abeab12cbe9be0f16d4a7b15017937a5de66eb67a38073de7eb72
+    v1.18.3: 6aac8853028a4f185de5ccb5b41b3fbd87726161445dee56f351e3e51442d669
    v1.18.2: bc13d29b58300c328f0078c7f72e37e1254c4303277348862af1e7f2b356b9e3
    v1.18.1: 4c5737235e62a5bb0b5d3f51939ccd255ebda376d75941222b25241251b67fbc
    v1.18.0: 3a90e7abf9910aebf9ef5845918c665afd4136a8832604ccfabca2defb35ce0f
+    v1.17.12: 196263deb2605507ff137b3edea39a914e7047f4c4de0a681c4a6f8dcfa5fe65
+    v1.17.11: 71bcc8443a6e6f226727ea007fdc4b96327e302d1724a15ce2bffc2d94a3dac6
+    v1.17.9: 3b6cdfcd38a646c7b553821ef9bb67e93541da658305c00705e6ab2ba15e73af
+    v1.17.8: b39081fb40332ae12d262b04dc81630e5c6550fb196f09b60f3d726283dff17f
+    v1.17.7: a6b66c94a37dd6ae830a9af5b9200884a2c0af868096a3c2553b2e876723c2a2
+    v1.17.6: 4b7fd5123bfafe2249bf91ed83469c2655a8d3295966e5fbd952f89b64b75f57
    v1.17.5: c5fbfa83444bdeefb51934c29f0b4b7ffc43ce5a98d7f957d8a11e3440055383
    v1.17.4: f3a427ddf610b568db60c8d47565041901220e1bbe257614b61bb4c76801d765
    v1.17.3: a5c2349c61771f8bf9f80feb174f7e9d9a6c9e79559758ea538ed3dead07bdcb
    v1.17.2: 33c6befab43ace4c4e89eab9c45d0cea5432f3cea4beaa956c786fe521f844bb
    v1.17.1: ffd04d1934c193fa63b3fc7d285d3646ed215f07f726390eefb0913b810716c3
    v1.17.0: c2af77f501c3164e80171903028d35c632366f53dec0c8419828d4e55d86146f
+    v1.16.15: 3d8337e2e0b613688d3828803988adeafedc9451c785478951002bf627beb36b
+    v1.16.14: a094022d630a1fb34080c1e317e698a9adb9452b0c821c96d96511e8b4489956
+    v1.16.13: a88c0e9f8c4b5a2e91c2c4a8d772cc65ca3a0eb5d477cbce06fbf82d3e50c158
+    v1.16.12: fbc8c16b148dbb3234a3e13f80e6c6736557c10f8c046edfb1dc5337fe2dd40f
+    v1.16.11: edab125cf34c5e95cd883b52385861247cb68ed45605e0ba0774dcf55bde9519
+    v1.16.10: 82b38f444d11c2436040165b1addf46d0909a6daec9133cc979678835ef8e14b
    v1.16.9: 79e7a1500e154b53087cf7895a710d081d2c357bd34d05362edf230e3c269e63
    v1.16.8: 4573da19fed14c84f4434ab7cbedf5ded4bf89710c078d58c0703cf2332df198
    v1.16.7: f49755b06848914c2729353d3580199a70ec8d732609660e90214b4f48ff4398
@ -220,15 +274,33 @@ kubelet_checksums:

 kubectl_checksums:
  arm:
+    v1.18.9: 8f49ade7875aaca82f7471901963796815b786d5437e6af0ae4d6d784dc92c08
+    v1.18.8: 21769e01e17f3809d0e9188a88a71fb1f479dfeeb22590e56006d5dbb3689605
+    v1.18.6: 1b6668bae432a3c6034f2fe83e72b2db5ea794a02121cf0a22be7f4bd8f460bf
+    v1.18.5: 5fc8dc6e3d09ceaf900dd06b9af3a7abb291293cea5219aea577bad852aa84b1
+    v1.18.4: 9617b1a929aad7e3bf9f1151f83548e5e3f89175f5d3f961733b8b0ec2e376c4
+    v1.18.3: 1816364467b98e7ae52731f593780f392d6835d33db5b12a671abfffb72a4eac
    v1.18.2: 353d61297cfbf01ed3f72b7df1658110c065355d670556ea3bdbf0d1b2824aea
    v1.18.1: 896c90b1b9d88e121876d93718591f3ecbab880b304767806c6c9fcb3b145805
    v1.18.0: 34fc6d16c2f535ed381b5fd3a4d40b642fff4f9ff95f8250b8043a29b8c062b9
+    v1.17.12: e3bfbb4a82183c094a6538ffe4e837856cb3849ed1348d23830f0945852a81e4
+    v1.17.11: e8178745c3010a57de068138e63bf19087ca4f83acfdc1df82f7844ce73cf3be
+    v1.17.9: 4bcad42cb8721bcb636f88e02f143fa9e2ad8141a37025f7622bd04516dab391
+    v1.17.8: e1a75ee55e1270583143422cc611547623aeef2c69689354c69b0b8f445cf6ba
+    v1.17.7: 1b862c79333b7edee64f0317f8c5de8699f99b00709734e3341d41cca3b8f29b
+    v1.17.6: 2ad9897b84dd503c963ff790ce092aeb4c8e78ac64b7986a6c6ed1c601255419
    v1.17.5: 470139a2ca98a85ab89210d07dc733d457d48a8419bbf038ee7e55276e2b5c35
    v1.17.4: bec3f4163231d4df62ef75b1e435f646b576bfeff08a5e635a033c8223fb4c52
    v1.17.3: 740e17e7fa2b6aed243e690cdb939f040aefe644a485429ed42b2b1fa7eac813
    v1.17.2: 152e5b5e1a744ad8e4860bef212462750e0a38856990d6a4d0b3418bedb5346f
    v1.17.1: a1e580e9140536c4a370c207ee66481cfe8d8876dc9021755a9d20232a97033d
    v1.17.0: 594b3e2f89dca09d82b176b51bf6c8c0fa524ed209c14ec915c9b36fa876601d
+    v1.16.15: 23436a1b74b535338543cb6e8a202d2cf62881479971393fabff0e692a1ec677
+    v1.16.14: be69bf086b263498a4e80e4df3b73315af0c7b9ac918a3af8946a8dff23d2d56
+    v1.16.13: 5fd835f53e20a3fbee21961eee0f492a205cc08df8221538185e57dca85ed82f
+    v1.16.12: 0f39db272aa24e12f5842bd30623801e677232ec42d469935d3ecf0040a72970
+    v1.16.11: f0d50fbc930d97220d5dccea309025f42341b7a99b113032d05860f03c7f33dc
+    v1.16.10: f196dfd5592f1d4cbe92b3e7a7de2307bbbf9cec0a09d4eb22b645650ed79af7
    v1.16.9: 1d5627c9e186c6f3b501045e1328f54925d2ced852f93baf2e89a342fa85e788
    v1.16.8: 803c3197845fd5f97c99bcba6e7a0af223128442deccca92df2d267d545db166
    v1.16.7: 588ac4f1387442d0b0d8928080a7f42c1196ae9b0b9a3d6a25b048df26926e26
@ -251,15 +323,33 @@ kubectl_checksums:
    v1.15.1: 17f1ba668d98f6d57b2494273a81b49c35aecc3d13a3de36cef390e0bad60ee0
    v1.15.0: 3ea07f7fa18309a630a9f317582de742befbf8efdcb22300a192941b4075c16f
  arm64:
+    v1.18.9: 9f466ff8d40097914a1ded0288ef8b9eb6f4ad39a9990cb2c0f8d1a710013a4f
+    v1.18.8: 9046c4086528427462544e1a6dcbe709de4d7ae44d1a155375de330fecd067b1
+    v1.18.6: 7b3d6cc019747a7ee5f6cc2b187423daaac4e153140cb290e60d316c3f456430
+    v1.18.5: 28c1edb2d76f80e70e10fa8cd2a30b9fccc5f003d8b3e853535d8317db7f424a
+    v1.18.4: 61c5004f6e9040163bc09459a11fd17b0f9ff55d7ba8f9b1e89368b5f2cdf072
+    v1.18.3: fc4479d1f7e58e6c8f40430a35f6b09b6f582909f69968e424fc20640ac45daf
    v1.18.2: 8d4bd6a716e32187e03c5998b4d9570f3b2eb9fb041ac9ed6e9728f04935c2fb
    v1.18.1: 39e9645c6bed1e7340b3d764db983b9fc35326b11325fd509c3660c9f55469ed
    v1.18.0: 0de307f90502cd58e5785cdcbebeb552df81fa2399190f8a662afea9e30bc74d
+    v1.17.12: 22f383cef1a429cab5d4c98df6704295722044613dbea49c306e931d383d723f
+    v1.17.11: 8eb18f37148d2786205bc70dded66c173df7517577e5ae024a19e72400263ed2
+    v1.17.9: 4d818e97073113eb1e62bf97d63876757be0f273c47807c09f34511155e25afd
+    v1.17.8: 4dfd36dbd637b8dca9a7c4e789fb3fe4ca420062c90d3a872ae751dfb9777cb6
+    v1.17.7: 00c71ceffa9b50af081d2838b102be49ca224a8aa928f5c948b804af84c58818
+    v1.17.6: ceccf6ef3e0ac523cb75d46d1b4979ae1f8cf199926244a9d828cb77f024e46b
    v1.17.5: 160d1198a6da3eb082e197e368ba86c2acce435e073e9f3ee271aa59c7fb47d6
    v1.17.4: 95867f3c977b1f754223b95dbb04a9ff45613529e9e4691ffa45c6b214f9fd4f
    v1.17.3: d007a212240fef9fee30c59b4d4203bbc463d334f679c4d0d1af521b7e2c42e6
    v1.17.2: 29c36d5866a76ca693a255567ac26d7558c1f02e6b840895093e47afe06594d9
    v1.17.1: 4208be10e2c12b67e71219cd39b0b2ab065d4ec1b26e19c5da88cb8ebc64ea2f
    v1.17.0: cba12bfe0ee447b06f00813d7d4ba3fbdbf5116eccc4d3291987044f2d6f93c2
+    v1.16.15: 74719f137dc6d589a3b8a667bcb0f3c57eebd8f050dd2f7ad5b59ceb892a7b99
+    v1.16.14: 82450e2ce9a3fdac0f3103777ee03ae4b64d7e35bf437fd4fa3db86e9cf875b7
+    v1.16.13: 22013ef1558da3d8da2bf725e0441631029ab404816e765a240456906665e196
+    v1.16.12: 7f493dcf9d4edfeea68284c4cd7c74383be23f24e9aefd59c08dc37bc20b46db
+    v1.16.11: a6bc2f6b099c19fd1f0748ebb8cc90a710e2af32f1c245c08c92e77db609ce3b
+    v1.16.10: 94418c3817fdeac8263cdf019f78313a8a45b539e96a1962d11a308b75a18438
    v1.16.9: c957a8a346b7e83c33b8ed6386b8d3e942e34bbc8794bdca33f7304977fc377e
    v1.16.8: d08aab5f02db63690672e5d9052659589301323c010d90734788d5332ac99daa
    v1.16.7: 404289fded50daeba0f66e36275498164f17ab121f6cd3cfb1bea6b85914c6e5
@ -282,15 +372,33 @@ kubectl_checksums:
    v1.15.1: 1f5ad15f0522c5038ea014f4b6987a7a67d68585c1fb158349619b552c027a8c
    v1.15.0: db05f4c1799e019d5ebc51737f31132c191ffce13c516fa758e7137173abd855
  amd64:
+    v1.18.9: 6a68756a2d3d04b4d0f52b00de6493ba2c1fcb28b32f3e4a0e99b3d9f6c4e8ed
+    v1.18.8: a076f5eff0710de94d1eb77bee458ea43b8f4d9572bbb3a3aec1edf0dde0a3e7
+    v1.18.6: 62fcb9922164725c7cba5747562f2ad2f4d834ad0a458c1e4c794cc203dcdfb3
+    v1.18.5: 69d9b044ffaf544a4d1d4b40272f05d56aaf75d7e3c526d5418d1d3c78249e45
+    v1.18.4: 5fea9ad294ea73f952243178db5340dc29c14ad96aed3f92a18deedb73f221ec
+    v1.18.3: 6fcf70aae5bc64870c358fac153cdfdc93f55d8bae010741ecce06bb14c083ea
    v1.18.2: 6ea8261b503c6c63d616878837dc70b758d4a3aeb9996ade8e83b51aedac9698
    v1.18.1: f5144823e6d8a0b78611a8d12e7a25202126d079c3a232b18f37e61e872ff563
    v1.18.0: bb16739fcad964c197752200ff89d89aad7b118cb1de5725dc53fe924c40e3f7
+    v1.17.12: d6472a5f5d3ff125b4a1aa5cefd6197faadfb578c36f639240a04df1a71597e1
+    v1.17.11: 002d640a12d6fad48f7d8a5c56cb4ff656397caf3511f50426d9c69c94b2d137
+    v1.17.9: 2ca83eecd221bedf3eceb0ccfcf45bb2e27950c382c2326211303adb0a9c4232
+    v1.17.8: 01283cbc2b09555cbf2a71c162097552a62a4fd48a0a4c06e34e9b853b815486
+    v1.17.7: 7124a296518edda2ae326e754aec9be6d0ac86131e6f61b52f5ecaa413b66ae4
+    v1.17.6: 5e245f6af6fb761fbe4b3ac06b753f33b361ce0486c48c85b45731a7ee5e4cca
    v1.17.5: 03cd1fa19f90d38005148793efdb17a9b58d01dedea641a8496b9cf228db3ab4
    v1.17.4: 465b2d2bd7512b173860c6907d8127ee76a19a385aa7865608e57a5eebe23597
    v1.17.3: ae8627adb1f0ae7bdb82ffd74a579953b8acdcd4b94aeefc7569078c3d7994c6
    v1.17.2: 7732548b9c353114b0dfa173bc7bcdedd58a607a5b4ca49d867bdb4c05dc25a1
    v1.17.1: a87a0acdc67d066bc331cb96c7fd29a883d67a41beeef538a0bd2878872ebad9
    v1.17.0: 6e0aaaffe5507a44ec6b1b8a0fb585285813b78cc045f8804e70a6aac9d1cb4c
+    v1.16.15: e8913069293156ddf55f243814a22d2384fc18b165efb6200606fdeaad146605
+    v1.16.14: c9cb4652768771e1e0abffb6bdb9cc0e38814913f0ab6afd2462f5e967734ad3
+    v1.16.13: ab861ec3ec347062bd1b87f8d78d15cd1ce251e74c5fe662e434056962d2a2c9
+    v1.16.12: db72e5c90de59e1bf287bef55eaf0b603c8d74b3dc552f356ccc02b08c2eb348
+    v1.16.11: fe65c523d52dbcd1973069a96ca4fd2d1c81ea941d79864114fd5e5c75549012
+    v1.16.10: 246d36e4ce67e74e95ff2ba578b9189f58e5def0e8830a24cd30fa3cf279742f
    v1.16.9: 0f3a6618a2e7402b11a1d9b9ffeff3ba0c6765dc361815413ce7441799aecf96
    v1.16.8: 1d8602496ca4b843824a9746206509991eb8d30b5bb8436b36a02718729934ed
    v1.16.7: c31ca51b526489cd929be71fc1dc9c3cc24b6df5641b3505b467bac51862047d
@ -315,15 +423,33 @@ kubectl_checksums:

 kubeadm_checksums:
  arm:
+    v1.18.9: 026cd1ec3b75703994254ae44998a544f46723b424775218f90c07754bb42bb6
+    v1.18.8: 52ec1a5d8a6826762c112d55734e35cf895a02e746b8d6ca4a9c942289aab077
+    v1.18.6: 84abadc0abc01970ee73bf25078b2120a0725e4afcb9bb8c0d03077c978d7452
+    v1.18.5: 461641c8fb8db2afe6e103aca925a4ef9d161dcae08a96fc24674b0ea0122e04
+    v1.18.4: 0a8a021cb3d18295f53843b1ab7d2d8bf9b861d5d6bd160f24717d22aa5a8fa7
+    v1.18.3: 88b8004dcfbf8862e5ae4dadcd4e4ef86c91211e48cd45922d5a18634b06d1b3
    v1.18.2: c3558beca26c1b970cee8419dcf24f9812483f6ef384cea9a704491bc3af1e2c
    v1.18.1: 4f919ad7215209dee97ea4c61668e44a2cce8f575b9cf4032e47f0c377924854
    v1.18.0: 0f05bd526bb38be11459675c69bc882a2d3e583e48339fab49b620d292c2433e
+    v1.17.12: c18a1c4aa3788c8e860a8653987bd50df9f2da70e15e5a4e1cce6f9a4bee4831
+    v1.17.11: df79e696668290091cf93b4c68ff614b3886cecbe40dbf76fdfff799ba41901d
+    v1.17.9: 69c28a8e35394cf72b3926bb557724b7d0c5bbd07a22fdd9c207b943b4c617b3
+    v1.17.8: 1e8e653a07438131126f62b853b442356b341d2950f0d7c30d2a96e773a54611
+    v1.17.7: 47c911a7deff993e654da1e0644fe627e496292d7a7a5f43f33fa4cde6b6856d
+    v1.17.6: a12f4281d018a7d53611cb1c0c537cd8f82dc01f3e16c16513622c1d6c9db658
    v1.17.5: ae2b66de65a6a435ff06ea8e542904e92c5eec0c42c2e57905a2a31a52106ca1
    v1.17.4: d22dd143947aa442812b325f36d48929506ea8416230213ffb83c29c1c1222f5
    v1.17.3: fc94d273927bc7e1dce91518133492f4e76aead6e795338317281fb0c6b6445e
    v1.17.2: c0a74989da367d9c11b25d4fbd90e8d3d1a013a63c9be7bbce61b320715c1a83
    v1.17.1: 501d1bacb863713dd9d0101d0021b0227869c4b1b9e903f6498333c613d384e1
    v1.17.0: 5fcf1234d89bc2a364c53b76b36134fc57278b456138d93c278805f2c9b186f1
+    v1.16.15: 68cec658085d799d6db5a828f9e49005e5f88130720ec916af5c17c557da586d
+    v1.16.14: 6c44968fbb3480e10eb88117a40abd8448153eb2445faf6d3a4b6869a80a29d4
+    v1.16.13: c18f6682f8f5787129b0a0903766822265c0e8052a3bd60a3594625615e6f928
+    v1.16.12: 0de9916e2117304d1f112a6d2bd53f08043f81329c2c12ca17635b98c88bc6a6
+    v1.16.11: 9489861c4856d9b0fdf7124a2bd38c01f29ecc4a83c1b602083e1cc739040d10
+    v1.16.10: e7c09d060fcbe5948ea43fa9b0465ea2c0fd91de76b403530df23062e3b44341
    v1.16.9: fb2b48e7a866a09611d825fc122f6bc2b04473b0ccb06682436effd2351ad425
    v1.16.8: df7737c2ffbd6001fabbed7c84e9ab4f483f750aff5e6052c245a7978ab203af
    v1.16.7: 66dfa31f5ebd96b3f3a0243533772d68602d36c0e6c0114f4c004cd9c4bed481
@ -346,15 +472,33 @@ kubeadm_checksums:
    v1.15.1: 855abd520291dcef0577a1a2ef87a70f522fd2b22603a12abcd86c2f7ec9c022
    v1.15.0: 9464030a1d4e101de5f47348f3514d5a9eb95cbce2e5e31f53ada1ca485cf75e
  arm64:
+    v1.18.9: c17e29b8cec1892b6cd72aed1af6d9abfd39816c222d3cc5c97c6637a284162d
+    v1.18.8: 71f6d95f165a9e8066c6f299217af779829ab3d798f6130caf6daa4784dc0464
+    v1.18.6: df5a3d7c70c3f8221d57093c5cb17558aad6e65725d7a096c6620302fbf64730
+    v1.18.5: 0e2a9de622177015c2514498382b0d821ac8f71c7ed5f02e5684d456ff3c0e4d
+    v1.18.4: 67feef5289663ac1bf7c3ab6bdc2d5ac2f24e9ca5ddad82129fd8ea1f9c8b747
+    v1.18.3: 6a6fda8e2abdaed05f9df16528c8c0ae59cbe89fbda467cce204bf548965863f
    v1.18.2: e5a1f738443c15f5f8f3b316c6c7f8038f84f24b5d4bf2eef5bee39ca208952a
    v1.18.1: 0cb6589d9b4c09b007eae977ab8a185fc4140eda886408dced4f500a508e4e83
    v1.18.0: 2ef1785159c80a9acd454a1c8be3c6b8db2260200b22e4359426e709ff786d01
+    v1.17.12: fb2f090c4bd62229a738b8e1faa81bd3b01252a49351464a1e1be9fe689ab959
+    v1.17.11: 0875b49d7eacb80db3b9a375086a091304420c875ae8b54191ed9aa0af4c54e3
+    v1.17.9: b56dc03177636fdafb4f8ab329d087b804cb7395c142f76e8246e86083c6d750
+    v1.17.8: 5a52e7d0306890e68ed66fc47ecd70bf14628c70527442fd0cd2973dbde7064c
+    v1.17.7: 6c8622adf5a7a2dfc66ebe15058353b2e2660b01f1e8990bab7a9c7fca76bccb
+    v1.17.6: b9f20f98aeecc7b011727ff8be9008a8229cdbea6d3dd93f782622c306306288
    v1.17.5: 6f004152ca1f60bb6ac7446e2c317957df5cff5ac55b60c08ce7869792dc4196
    v1.17.4: 20e1e095f8c46e5dba6366eec162a40b22cd7639f32e83743afef3c0daafd127
    v1.17.3: 92d584c2ff83790830384159fbf6d04798eea002d6315923657fd6f74c80f092
    v1.17.2: 091864574d38d3e30ed57734419b55d0957f39291d6f573ff8fffc8d474fb9ec
    v1.17.1: c640eb50406962628ac6e31fd840506a360b5d9c57d14007d0eaada28c49d64f
    v1.17.0: 0b94d1ace240a8f9995358ca2b66ac92072e3f3cd0543275b315dcd317798546
+    v1.16.15: f7cc47f8ec5eca6f72d90518ef91a08549a01ddc46a6e2c7b03756d83b8ecc8b
+    v1.16.14: 25c1b3a737fc8ca7037a80c89044d248dffa834e2f334b7bafdcc0dcb9ce914d
+    v1.16.13: e574ede80b3eacd197376edb1c04d950c8af62a036a049d16753615752b2d8be
+    v1.16.12: 67f675f8fb1ff3af56ca0a976323a65cabc35efa53b7896146684b8f53990741
+    v1.16.11: 21ae6f9275525130ec051ddbd09932f7baee2ab85b83f884a879a7b005f9fcb6
+    v1.16.10: 568d5be1cad4d2b2d4811055f2ec9fc478fd827d77d02b1ea1aa916eada7c32a
    v1.16.9: 2045b51d08ca4f1ca646edf8ed716b79200ac587d08b4281b03ca0297ecb01e5
    v1.16.8: 2300e2a7dc16512595c7aebc486799239039d33f33db2d085550d1f2d5f3129b
    v1.16.7: 90b69bdebefea6496335485d24bd9829e16029bbc7aa771749044cecd1a60360
@ -377,15 +521,33 @@ kubeadm_checksums:
    v1.15.1: 44fbfad0f1026d249fc4f365f1e9562cd52d75360d4d1032731122ba5a4d57dc
    v1.15.0: fe3c79070814fe847a23209b1027672fe5c5e7e5c9611e329225058926836f96
  amd64:
+    v1.18.9: 3f7f61e0fe3de43f5b345343f85d7ba5145737efb80974baa6076965f3a6963e
+    v1.18.8: 27c8f4d4398d57762998b157d35802a36a7ea9b2b6f9a363c397a9d65b4f3c89
+    v1.18.6: 11b4180b9f82a8b6bb30250e3d7341b104521f3b654076b8569853ec9451b2a9
+    v1.18.5: e428fc9d1cf860090346a83eb66082c3be6b6032f0db9e4f8e6d52492d46231f
+    v1.18.4: cec00c2629805b660b5f41b13292dfe75cbd3803e57a1ded53def912fedb1a22
+    v1.18.3: a60974e9840e006076d204fd4ddcba96213beba10fb89ff01882095546c9684d
    v1.18.2: 290bb6acb12c844f76affbab1ce374903bd97c4f19ac8cd3e6fdb7208d638ac8
    v1.18.1: fdb194647048f3e3ebdc93613b21a5b678fcbe0d212d08c0d56758d1bf2d2c85
    v1.18.0: 0261331c2ea718c0cd39114871aa098f1b4685f6101cb78cc880f645e72d0654
+    v1.17.12: 214f547788410a643d57aee8215d99f44ebe59967e973f83b024e2ba7832593f
+    v1.17.11: 328ca55af305fef59e72544d0faf76167b5b5dc5779cec17c670961e6529d907
+    v1.17.9: 5ef1660d3d56e93e3d87d6a7028aa64745984be0b0678c45c32f66043b4d69b4
+    v1.17.8: c59b85696c4cbabe896ba71f4bbc99e4ad2444fcea851e3ee740705584420aad
+    v1.17.7: 9d4b97e93ddb204798b91fec063743e218c92b42798779b5248a49e1476226e2
+    v1.17.6: d4cfc9a0a734ba015594974ee4253b8965b95cdb6e83d8a6a946675aad418b40
    v1.17.5: 9bd2fd1118b3d07d12e2a806c04bf34d99e79886c5318ddc003ba38f30da390c
    v1.17.4: 3cdcffcf8a1660241a045cfdfed3ebbf7f7c6a0840f008e2b049b533bca5bb8c
    v1.17.3: e34e3193a1161aea7269cee3f115e86ff71f01702a1c15fa0f71103bf2dba304
    v1.17.2: 33a1d8e3cea2bdbb9fa9cb257c516289ee50d957fcb6d7b35919f5f0e6ca2f41
    v1.17.1: 11bd31833dab9adb5b53398772dd1582264c3d1757cb3395e691d6a7379081ec
    v1.17.0: 0d8443f50fb7caab2e5e7e53f9dc56d5ffe55f021ec061f2e2bcba0481df5a48
+    v1.16.15: 2d00f583be1dcc0540122e1d3855d7074b6380176aa50673903334d2d612b10e
+    v1.16.14: afb935df01910d1e7de34a9cbe7cbc885d3345dbba571d8408b03eddd27bd0b2
+    v1.16.13: 3ddce3fb919f1e8b0a3e0a1ae1d20c9af0fd4a7d731be1e818597b3ecdb49023
+    v1.16.12: bb4d0f045600b883745016416c14533f823d582f4f20df691b7f79a6545b6480
+    v1.16.11: a40ed479d89271f2d5514121a3ad3cb0a9bc90845d511c0c87b1c99bcca880f1
+    v1.16.10: 726d42c569f25078d03b758477f17f543c845aef2ff48acd9d4269705ca1aa9d
    v1.16.9: 99b3bd3a59e5832b2bfe3f3936ffd1f983e22913b32111684311d11fd2cefbf1
    v1.16.8: 58a74986af13b969abc8b471822f36f3fda71f95ed1c006f48c8d2ab88f8edf1
    v1.16.7: 018f7c2fae7a1ce7bc892edbea3ee2c493e4b023436be9f8f65fd392dc3e17b6
@ -415,9 +577,9 @@ etcd_binary_checksums:
  arm64: 170b848ac1a071fe7d495d404a868a2c0090750b2944f8a260ef1c6125b2b4f4
  amd64: dc5d82df095dae0a2970e4d870b6929590689dd707ae3d33e7b86da0f7f211b6
 cni_binary_checksums:
-  arm: 86a868234045837cb3f5d58a0a4468ff42845d50b5e87bd128f050ef393d7495
-  arm64: a7881ec37e592c897bdfd2a225b4ed74caa981e3c4cdcf8f45574f8d2f111bce
-  amd64: bd682ffcf701e8f83283cdff7281aad0c83b02a56084d6e601216210732833f9
+  arm: 28e61b5847265135dc1ca397bf94322ecce4acab5c79cc7d360ca3f6a655bdb7
+  arm64: 43fbf750c5eccb10accffeeb092693c32b236fb25d919cf058c91a677822c999
+  amd64: 994fbfcdbb2eedcfa87e48d8edb9bb365f4e2747a7e47658482556c12fd9b2f5
 calicoctl_binary_checksums:
  arm:
    v3.13.2: 0
@ -462,13 +624,13 @@ etcd_image_repo: "{{ quay_image_repo }}/coreos/etcd"
 etcd_image_tag: "{{ etcd_version }}{%- if image_arch != 'amd64' -%}-{{ image_arch }}{%- endif -%}"
 flannel_image_repo: "{{ quay_image_repo }}/coreos/flannel"
 flannel_image_tag: "{{ flannel_version }}"
-calico_node_image_repo: "{{ docker_image_repo }}/calico/node"
+calico_node_image_repo: "{{ quay_image_repo }}/calico/node"
 calico_node_image_tag: "{{ calico_version }}"
-calico_cni_image_repo: "{{ docker_image_repo }}/calico/cni"
+calico_cni_image_repo: "{{ quay_image_repo }}/calico/cni"
 calico_cni_image_tag: "{{ calico_cni_version }}"
-calico_policy_image_repo: "{{ docker_image_repo }}/calico/kube-controllers"
+calico_policy_image_repo: "{{ quay_image_repo }}/calico/kube-controllers"
 calico_policy_image_tag: "{{ calico_policy_version }}"
-calico_typha_image_repo: "{{ docker_image_repo }}/calico/typha"
+calico_typha_image_repo: "{{ quay_image_repo }}/calico/typha"
 calico_typha_image_tag: "{{ calico_typha_version }}"
 pod_infra_image_repo: "{{ kube_image_repo }}/pause"
 pod_infra_image_tag: "{{ pod_infra_version }}"
@ -493,11 +655,11 @@ contiv_etcd_init_image_repo: "{{ docker_image_repo }}/ferest/etcd-initer"
 contiv_etcd_init_image_tag: latest
 contiv_ovs_image_repo: "{{ docker_image_repo }}/contiv/ovs"
 contiv_ovs_image_tag: "latest"
-cilium_image_repo: "{{ docker_image_repo }}/cilium/cilium"
+cilium_image_repo: "{{ quay_image_repo }}/cilium/cilium"
 cilium_image_tag: "{{ cilium_version }}"
-cilium_init_image_repo: "{{ docker_image_repo }}/cilium/cilium-init"
+cilium_init_image_repo: "{{ quay_image_repo }}/cilium/cilium-init"
 cilium_init_image_tag: "2019-04-05"
-cilium_operator_image_repo: "{{ docker_image_repo }}/cilium/operator"
+cilium_operator_image_repo: "{{ quay_image_repo }}/cilium/operator"
 cilium_operator_image_tag: "{{ cilium_version }}"
 kube_ovn_container_image_repo: "{{ alauda_image_repo }}/alaudak8s/kube-ovn"
 kube_ovn_container_image_tag: "{{ kube_ovn_version }}"
@ -514,7 +676,8 @@ haproxy_image_tag: 1.9
 # Coredns version should be supported by corefile-migration (or at least work with)
 # bundle with kubeadm; if not 'basic' upgrade can sometimes fail
 coredns_version: "1.6.5"
-coredns_image_repo: "{{ docker_image_repo }}/coredns/coredns"
+coredns_image_repo: "{{ kube_image_repo }}/coredns"
+
 coredns_image_tag: "{{ coredns_version }}"

 nodelocaldns_version: "1.15.12"
@ -524,10 +687,8 @@ nodelocaldns_image_tag: "{{ nodelocaldns_version }}"
 dnsautoscaler_version: 1.7.1
 dnsautoscaler_image_repo: "{{ kube_image_repo }}/cluster-proportional-autoscaler-{{ image_arch }}"
 dnsautoscaler_image_tag: "{{ dnsautoscaler_version }}"
-test_image_repo: "{{ docker_image_repo }}/library/busybox"
+test_image_repo: "{{ kube_image_repo }}/busybox"
 test_image_tag: latest
-busybox_image_repo: "{{ docker_image_repo }}/library/busybox"
-busybox_image_tag: 1.31.1
 helm_version: "v3.1.2"
 helm_image_repo: "{{ docker_image_repo }}/lachlanevenson/k8s-helm"
 helm_image_tag: "{{ helm_version }}"
@ -933,15 +1094,6 @@ downloads:
    groups:
      - kube-master

-  busybox:
-    enabled: "{{ kube_network_plugin in ['kube-router'] }}"
-    container: true
-    repo: "{{ busybox_image_repo }}"
-    tag: "{{ busybox_image_tag }}"
-    sha256: "{{ busybox_digest_checksum|default(None) }}"
-    groups:
-      - k8s-cluster
-
  testbox:
    enabled: false
    container: true
--- a/roles/kubernetes-apps/ansible/templates/dashboard.yml.j2
+++ b/roles/kubernetes-apps/ansible/templates/dashboard.yml.j2
@ -17,9 +17,18 @@
 #
 # Example usage: kubectl create -f <this_file>

+{% if dashboard_namespace != "kube-system" %}
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: {{ dashboard_namespace }}
+  labels:
+    name: {{ dashboard_namespace }}
+{% endif %}
+
 ---
 # ------------------- Dashboard Secrets ------------------- #
-
 apiVersion: v1
 kind: Secret
 metadata:
@ -178,7 +187,7 @@ spec:
            - --auto-generate-certificates
 {% endif %}
 {% if dashboard_skip_login %}
-          - --enable-skip-login
+            - --enable-skip-login
 {% endif %}
            - --authentication-mode=token{% if kube_basic_auth|default(false) %},basic{% endif %}
            # Uncomment the following line to manually specify Kubernetes API server Host
--- a/roles/kubernetes/master/handlers/main.yml
+++ b/roles/kubernetes/master/handlers/main.yml
@ -93,7 +93,8 @@

 - name: Master | wait for kube-scheduler
  uri:
-    url: http://localhost:10251/healthz
+    url: https://localhost:10259/healthz
+    validate_certs: no
  register: scheduler_result
  until: scheduler_result.status == 200
  retries: 60
@ -101,7 +102,8 @@

 - name: Master | wait for kube-controller-manager
  uri:
-    url: http://localhost:10252/healthz
+    url: https://localhost:10257/healthz
+    validate_certs: no
  register: controller_manager_result
  until: controller_manager_result.status == 200
  retries: 60
@ -111,8 +113,6 @@
  uri:
    url: "{{ kube_apiserver_endpoint }}/healthz"
    validate_certs: no
-    client_cert: "{{ kube_apiserver_client_cert }}"
-    client_key: "{{ kube_apiserver_client_key }}"
  register: result
  until: result.status == 200
  retries: 60
--- a/roles/kubernetes/master/tasks/kubeadm-upgrade.yml
+++ b/roles/kubernetes/master/tasks/kubeadm-upgrade.yml
@ -1,7 +1,7 @@
 ---
 - name: kubeadm | Check api is up
  uri:
-    url: "https://{{ ip | default(fallback_ips[inventory_hostname]) }}:6443/healthz"
+    url: "https://{{ ip | default(fallback_ips[inventory_hostname]) }}:{{ kube_apiserver_port }}/healthz"
    validate_certs: false
  when: inventory_hostname == groups['kube-master']|first
  register: _result
--- a/roles/kubernetes/node/tasks/main.yml
+++ b/roles/kubernetes/node/tasks/main.yml
@ -189,7 +189,7 @@

 - name: Test if openstack_cacert is a base64 string
  set_fact:
-    openstack_cacert_is_base64: "{% if openstack_cacert | b64decode %}true{% else %}false{% endif %}"
+    openstack_cacert_is_base64: "{% if openstack_cacert | search ('^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?$') %}true{% else %}false{% endif %}"
  when:
    - cloud_provider is defined
    - cloud_provider == 'openstack'
--- a/roles/kubernetes/preinstall/defaults/main.yml
+++ b/roles/kubernetes/preinstall/defaults/main.yml
@ -55,3 +55,6 @@ minimal_node_memory_mb: 1024
 minimal_master_memory_mb: 1500

 yum_repo_dir: /etc/yum.repos.d
+
+# number of times package install task should be retried
+pkg_install_retries: 4
--- a/roles/kubernetes/preinstall/handlers/main.yml
+++ b/roles/kubernetes/preinstall/handlers/main.yml
@ -9,18 +9,18 @@
    - Preinstall | restart kube-controller-manager crio/containerd
    - Preinstall | restart kube-apiserver docker
    - Preinstall | restart kube-apiserver crio/containerd
-  when: not ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"]
+  when: not ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"] and not is_fedora_coreos

 - name: Preinstall | update resolvconf for Container Linux by CoreOS and Flatcar
  command: /bin/true
  notify:
    - Preinstall | apply resolvconf cloud-init
    - Preinstall | reload kubelet
-  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"]
+  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"] or is_fedora_coreos

 - name: Preinstall | apply resolvconf cloud-init
  command: /usr/bin/coreos-cloudinit --from-file {{ resolveconf_cloud_init_conf }}
-  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"]
+  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"] or is_fedora_coreos

 - name: Preinstall | reload kubelet
  service:
--- a/roles/kubernetes/preinstall/tasks/0040-set_facts.yml
+++ b/roles/kubernetes/preinstall/tasks/0040-set_facts.yml
@ -72,12 +72,12 @@
      {%- if resolvconf|bool -%}/etc/resolvconf/resolv.conf.d/base{%- endif -%}
    head: >-
      {%- if resolvconf|bool -%}/etc/resolvconf/resolv.conf.d/head{%- endif -%}
-  when: not ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"]
+  when: not ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"] and not is_fedora_coreos

 - name: target temporary resolvconf cloud init file (Container Linux by CoreOS)
  set_fact:
    resolvconffile: /tmp/resolveconf_cloud_init_conf
-  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"]
+  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"] or is_fedora_coreos

 - name: check if /etc/dhclient.conf exists
  stat:
--- a/roles/kubernetes/preinstall/tasks/0060-resolvconf.yml
+++ b/roles/kubernetes/preinstall/tasks/0060-resolvconf.yml
@ -1,7 +1,7 @@
 ---
 - name: create temporary resolveconf cloud init file
  command: cp -f /etc/resolv.conf "{{ resolvconffile }}"
-  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"]
+  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"] or is_fedora_coreos

 - name: Add domain/search/nameservers/options to resolv.conf
  blockinfile:
@ -47,7 +47,7 @@
 - name: get temporary resolveconf cloud init file content
  command: cat {{ resolvconffile }}
  register: cloud_config
-  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"]
+  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"] or is_fedora_coreos

 - name: persist resolvconf cloud init file
  template:
@ -55,5 +55,5 @@
    src: resolvconf.j2
    owner: root
    mode: 0644
-  notify: Preinstall | update resolvconf for Container Linux by CoreOS
-  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"]
+  notify: Preinstall | update resolvconf for Container Linux by CoreOS and Flatcar
+  when: ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk"] or is_fedora_coreos
--- a/roles/kubernetes/preinstall/tasks/0070-system-packages.yml
+++ b/roles/kubernetes/preinstall/tasks/0070-system-packages.yml
@ -77,7 +77,7 @@
    state: latest
  register: pkgs_task_result
  until: pkgs_task_result is succeeded
-  retries: 4
+  retries: "{{ pkg_install_retries }}"
  delay: "{{ retry_stagger | random + 3 }}"
  when: not (ansible_os_family in ["CoreOS", "Coreos", "Container Linux by CoreOS", "Flatcar", "Flatcar Container Linux by Kinvolk", "ClearLinux"] or is_fedora_coreos)
  tags:
--- a/roles/kubespray-defaults/defaults/main.yaml
+++ b/roles/kubespray-defaults/defaults/main.yaml
@ -15,7 +15,7 @@ is_fedora_coreos: false
 disable_swap: true

 ## Change this to use another Kubernetes version, e.g. a current beta release
-kube_version: v1.17.5
+kube_version: v1.17.12

 ## The minimum version working
 kube_version_min_required: v1.16.0
--- a/roles/kubespray-defaults/tasks/no_proxy.yml
+++ b/roles/kubespray-defaults/tasks/no_proxy.yml
@ -26,6 +26,7 @@
 - name: Populates no_proxy to all hosts
  set_fact:
    no_proxy: "{{ hostvars.localhost.no_proxy_prepare }}"
-    proxy_env:
-      no_proxy: "{{ hostvars.localhost.no_proxy_prepare }}"
-      NO_PROXY: "{{ hostvars.localhost.no_proxy_prepare }}"
+    proxy_env: "{{ proxy_env | combine({
+      'no_proxy': hostvars.localhost.no_proxy_prepare,
+      'NO_PROXY': hostvars.localhost.no_proxy_prepare
+    }) }}"
--- a/tests/common/_docker_hub_registry_mirror.yml
+++ b/tests/common/_docker_hub_registry_mirror.yml
@ -0,0 +1,15 @@
+---
+docker_registry_mirrors:
+  - "https://mirror.gcr.io"
+
+containerd_config:
+  grpc:
+    max_recv_message_size: 16777216
+    max_send_message_size: 16777216
+  debug:
+    level: ""
+  registries:
+    "docker.io":
+      - "https://mirror.gcr.io"
+      - "https://registry-1.docker.io"
+  max_container_log_line_size: -1
--- a/tests/files/packet_centos7-crio.yml
+++ b/tests/files/packet_centos7-crio.yml
@ -0,0 +1,14 @@
+---
+# Instance settings
+cloud_image: centos-7
+mode: default
+
+# Kubespray settings
+deploy_netchecker: true
+dns_min_replicas: 1
+container_manager: crio
+
+# CRI-O requirements
+download_container: false
+etcd_deployment_type: host
+kubelet_deployment_type: host
--- a/tests/files/packet_centos8-calico.yml
+++ b/tests/files/packet_centos8-calico.yml
@ -7,6 +7,7 @@ mode: default
 kube_network_plugin: calico
 deploy_netchecker: true
 dns_min_replicas: 1
+dashboard_enabled: true

 # required / not autodetected for now
 calico_iptables_backend: "NFT"
--- a/tests/files/tf-ovh_coreos-calico.yml
+++ b/tests/files/tf-ovh_coreos-calico.yml
@ -1,5 +0,0 @@
---
-dns_min_replicas: 1
-deploy_netchecker: true
-
-# resolvconf_mode: host_resolvconf  # this is required as long as the coreos stable channel uses docker < 1.12
--- a/tests/files/tf-ovh_ubuntu18-calico.yml
+++ b/tests/files/tf-ovh_ubuntu18-calico.yml
@ -1,4 +0,0 @@
---
-dns_min_replicas: 1
-deploy_netchecker: true
-sonobuoy_enabled: true
--- a/tests/scripts/rebase.sh
+++ b/tests/scripts/rebase.sh
@ -1,9 +1,9 @@
 #!/bin/bash
 set -euxo pipefail

-# Rebase PRs on master to get latest changes
+# Rebase PRs on release-2.13 to get latest changes
 if [[ $CI_COMMIT_REF_NAME == pr-* ]]; then
  git config user.email "ci@kubespray.io"
  git config user.name "CI"
-  git pull --rebase origin master
+  git pull --rebase origin release-2.13
 fi
--- a/tests/scripts/testcases_run.sh
+++ b/tests/scripts/testcases_run.sh
@ -41,7 +41,8 @@ fi
 # Check out latest tag if testing upgrade
 test "${UPGRADE_TEST}" != "false" && git fetch --all && git checkout "$KUBESPRAY_VERSION"
 # Checkout the CI vars file so it is available
-test "${UPGRADE_TEST}" != "false" && git checkout "${CI_BUILD_REF}" tests/files/${CI_JOB_NAME}.yml tests/testcases/*.yml
+test "${UPGRADE_TEST}" != "false" && git checkout "${CI_BUILD_REF}" tests/files/${CI_JOB_NAME}.yml
+test "${UPGRADE_TEST}" != "false" && git checkout "${CI_BUILD_REF}" ${CI_TEST_REGISTRY_MIRROR}

 # Install mitogen ansible plugin
 if [ "${MITOGEN_ENABLE}" = "true" ]; then
@ -51,20 +52,20 @@ if [ "${MITOGEN_ENABLE}" = "true" ]; then
 fi

 # Create cluster
-ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "all:!fake_hosts" cluster.yml
+ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_REGISTRY_MIRROR} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "all:!fake_hosts" cluster.yml

 # Repeat deployment if testing upgrade
 if [ "${UPGRADE_TEST}" != "false" ]; then
  test "${UPGRADE_TEST}" == "basic" && PLAYBOOK="cluster.yml"
  test "${UPGRADE_TEST}" == "graceful" && PLAYBOOK="upgrade-cluster.yml"
  git checkout "${CI_BUILD_REF}"
-  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "all:!fake_hosts" $PLAYBOOK
+  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_REGISTRY_MIRROR} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "all:!fake_hosts" $PLAYBOOK
 fi

 # Test control plane recovery
 if [ "${RECOVER_CONTROL_PLANE_TEST}" != "false" ]; then
-  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "${RECOVER_CONTROL_PLANE_TEST_GROUPS}:!fake_hosts" -e reset_confirmation=yes reset.yml
-  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads -e etcd_retries=10 --limit etcd,kube-master:!fake_hosts recover-control-plane.yml
+  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_REGISTRY_MIRROR} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "${RECOVER_CONTROL_PLANE_TEST_GROUPS}:!fake_hosts" -e reset_confirmation=yes reset.yml
+  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_REGISTRY_MIRROR} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads -e etcd_retries=10 --limit etcd,kube-master:!fake_hosts recover-control-plane.yml
 fi

 # Tests Cases
@ -88,7 +89,7 @@ ansible-playbook -i ${ANSIBLE_INVENTORY} -e @${CI_TEST_VARS} --limit "all:!fake_

 ## Idempotency checks 1/5 (repeat deployment)
 if [ "${IDEMPOT_CHECK}" = "true" ]; then
-  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "all:!fake_hosts" cluster.yml
+  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_REGISTRY_MIRROR} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "all:!fake_hosts" cluster.yml
 fi

 ## Idempotency checks 2/5 (Advanced DNS checks)
@ -98,12 +99,12 @@ fi

 ## Idempotency checks 3/5 (reset deployment)
 if [ "${IDEMPOT_CHECK}" = "true" -a "${RESET_CHECK}" = "true" ]; then
-  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_VARS} -e reset_confirmation=yes --limit "all:!fake_hosts" reset.yml
+  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_REGISTRY_MIRROR} -e @${CI_TEST_VARS} -e reset_confirmation=yes --limit "all:!fake_hosts" reset.yml
 fi

 ## Idempotency checks 4/5 (redeploy after reset)
 if [ "${IDEMPOT_CHECK}" = "true" -a "${RESET_CHECK}" = "true" ]; then
-  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "all:!fake_hosts" cluster.yml
+  ansible-playbook ${ANSIBLE_LOG_LEVEL} -e @${CI_TEST_REGISTRY_MIRROR} -e @${CI_TEST_VARS} -e local_release_dir=${PWD}/downloads --limit "all:!fake_hosts" cluster.yml
 fi

 ## Idempotency checks 5/5 (Advanced DNS checks)
Author	SHA1	Message	Date
Rick Haan	cd832eadea	[2.13] Backport CI fix (#7119 ) * [2.14] fix ci (#7021) * fix flake8 errors in Kubespray CI - tox-inventory-builder * fix flake8 errors in Kubespray CI - tox-inventory-builder * Invalidate CRI-O kubic repo's cache Signed-off-by: Victor Morales <v.morales@samsung.com> * add support to configure pkg install retries and use in CI job tf-ovh_ubuntu18-calico (due to it failing often) * Switch Calico and Cilium image repos to Quay.io Co-authored-by: Victor Morales <v.morales@samsung.com> Co-authored-by: Barry Melbourne <9964974+bmelbourne@users.noreply.github.com> Conflicts: roles/download/defaults/main.yml * up vagrant box to fedora/33-cloud-base in cri-o molecule tests (cherry picked from commit `06ec5393d7`) * add Google proxy-mirror-cache for docker hub to CI tests (cherry picked from commit `d739a6bb2f`) * containerd docker hub registry mirror support * containerd docker hub registry mirror support * add docs * fix typo * fix yamllint * fix indent in sample and ansible-playbook param in testcases_run * fix md * mv common vars to tests/common/_docker_hub_registry_mirror.yml * checkout vars to upgrade tests (cherry picked from commit `4a8a52bad9`) * Exclude .git/ from shellcheck If a branch name contains '.sh', current shellcheck checks the branch file under .git/ and outputs error because the format is not shell script one. This makes shellcheck exclude files under .git/ to avoid this issue. (cherry picked from commit `e2467d87b6`) Co-authored-by: Hans Feldt <2808287+hafe@users.noreply.github.com> Co-authored-by: Sergey <s.bondarev@southbridge.ru> Co-authored-by: Kenichi Omichi <ken-oomichi@wx.jp.nec.com> * Switch some image from dockerhub to k8s.gcr (also increase pkg retries) (#6955) * Set packet_centos8-kube-ovn test to manual Signed-off-by: Rick Haan <rickhaan94@gmail.com> Co-authored-by: Etienne Champetier <champetier.etienne@gmail.com> Co-authored-by: Hans Feldt <2808287+hafe@users.noreply.github.com> Co-authored-by: Sergey <s.bondarev@southbridge.ru> Co-authored-by: Kenichi Omichi <ken-oomichi@wx.jp.nec.com> Co-authored-by: Florian Ruynat <16313165+floryut@users.noreply.github.com>	2021-01-21 06:49:04 -08:00
Florian Ruynat	d28a6d68f9	Add Kubernetes hashes 1.18.9/1.17.12 and set default (#6701 )	2020-09-18 02:08:45 -07:00
ahmadali shafiee	f4b6bce69f	[2.13] add missing 'Set up proxy environment' tasks (#6591 ) * Add proxy_env calculation to reset.yml (#6558) * * add proxy_env definition to remove_node.yml resolving #6430 (#6431) Co-authored-by: Florian Ruynat <florian234@hotmail.com> Co-authored-by: nniehoff <github@nickniehoff.net>	2020-09-17 02:46:45 -07:00
Florian Ruynat	8e53d81146	Add Kubernetes 1.16.15 hashes (#6626 )	2020-09-07 01:23:41 -07:00
Maxime Guyot	28ee071bd6	Update base image to v2.12.9 (#6492 )	2020-08-04 05:10:19 -07:00
Florian Ruynat	0ba1c8ff0a	Fix test if openstack_cacert is a base64 string (#6421 ) (#6475 )	2020-08-01 00:29:41 -07:00
Florian Ruynat	d6f859619c	Update hashes and set default version to 1.17.9 (#6435 )	2020-07-30 00:50:31 -07:00
Florian Ruynat	8a3bd6ba72	Move healthz check to secure ports (#6446 ) (#6457 )	2020-07-29 05:11:47 -07:00
Lovro Seder	d3fca7d47a	Fix shellcheck url (#6463 )	2020-07-28 06:03:07 -07:00
LucasBoisserie	587f7b33a7	Modify the populate no_proxy task to use a combine rather than relying on the hash_behaviour setting to be set to merge rather than replace (#6112 ) (#6346 ) Co-authored-by: Cody Seavey <seavey.cody@gmail.com>	2020-07-07 08:05:59 -07:00
Florian Ruynat	d3e8fd808e	Update hashes and set default version to 1.17.8 (#6336 )	2020-06-30 01:58:07 -07:00
Maxime Guyot	6bfc133d6c	[2.13] Update base image to v2.12.7 (#6295 ) * Use v2.12.7 base image for CI * Use 19.03.9 in localhost CI (#6201)	2020-06-22 04:32:39 -07:00
Florian Ruynat	3d6b9d6c15	Update hashes and set default to 1.17.7 (#6286 )	2020-06-18 23:43:58 -07:00
Etienne Champetier	39fa9503d9	[2.13] Bump CNI plugins to 0.8.6 (#6228 ) https://github.com/containernetworking/plugins/releases/tag/v0.8.6 https://github.com/kubernetes/kubernetes/issues/91507 Signed-off-by: Etienne Champetier <champetier.etienne@gmail.com> (cherry picked from commit `41b44739b1`)	2020-06-09 05:23:18 -07:00
spaced	051e08e31c	[2.13] Backport CRI-O bugfixes (#6230 ) * Enable crio 1.18 (#6197) * fix CRI-O repos for centos distributions (#6224) * fix CRI-O repos for centos distributions * fix CRI-O repos for centos distributions - revert workarounds * fix CRI-O repos for centos distributions - use https for centos repos * avoid 302 redirects for centos repos * Use OS packaging default value for apparmor_profile in crio.conf (#6125) Co-authored-by: jeanfabrice <github@bobo-rousselin.com>	2020-06-04 07:09:15 -07:00
bozzo	dd539cf360	Fix resolv.conf configuration for Fedora CoreOS. (#6155 )	2020-05-28 08:02:02 -07:00
Florian Ruynat	31094b1768	Add new Kubernetes version hashes and set default to 1.17.6 (#6183 ) * Add new Kubernetes version hashes and set default to 1.17.6 * Do not rebase against master when commiting a release branch	2020-05-25 02:43:11 -07:00
Florian Ruynat	1f4ef0e86e	[2.13] Backport recent bugfixes and mainly docker-cli issue (#6179 ) * Reorder tests in packet file (#6067) * Skip molecule tests for Ubuntu 18.04 (#6077) * Fix kubernetes-dashboard template identation (#6066) The `98e7a07fba` commit udpates the dashboard version to 2.0.0 but it enable skip login flag wasn't updated. This change updates its identation to avoid issues when dashboard_skip_login is enabled. * Disable OVH CI (#6114) * Create namespace when dashboard deployment uses customized namespace. (#6107) * Create namespace when dashboard deployment uses customized namespace. * Fix syntax. * Fix apiserver port when upgrading (#6136) * Fix docker fedora packages (#6097) * Match docker-cli version with docker-engine version (when available) * Disable upgrade jobs to allow release 2.13.1 (docker-cli bug) Co-authored-by: Maxime Guyot <Miouge1@users.noreply.github.com> Co-authored-by: Victor Morales <electrocucaracha@gmail.com> Co-authored-by: petruha <5363545+p37ruh4@users.noreply.github.com> Co-authored-by: Mateus Caruccio <mateus.caruccio@getupcloud.com>	2020-05-22 07:30:37 -07:00