C12s/c12s-kubespray
3
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

Calico upgrade path validation and old version cleanup (#6733)

Browse source 
* calico: add constant calico_min_version_required

and verify current deployed version against it.

* calico: remove upgrade support with data migration

The tool was used pre v3.0.0 and is no longer needed.

* calico: remove old version support from tasks

* calico: remove old ver support from policy ctrl

* calico: remove old ver support from node

* canal: remove old ver support

* remove unused calicoctl download checksums

calico_min_version_required is the oldest version that can be installed
Older versions can be removed.
This commit is contained in:
Hans Feldt 2020-09-24 18:04:06 +02:00 committed by GitHub
parent 50e8a52c74
commit 28073c76ac
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
19 changed files with 21 additions and 253 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
docs/calico.md
View file

@ -1,11 +1,5 @@
# Calico
N.B. **Version 2.6.5 upgrade to 3.1.1 is upgrading etcd store to etcdv3**
If you create automated backups of etcdv2 please switch for creating etcdv3 backups, as kubernetes and calico now uses etcdv3
After migration you can check `/tmp/calico_upgrade/` directory for converted items to etcdv3.
**PLEASE TEST upgrade before upgrading production cluster.**
Check if the calico-node container is running
```ShellSession
@ -20,24 +14,12 @@ The **calicoctl.sh** is wrap script with configured acces credentials for comman
calicoctl.sh node status
```
or for versions prior to *v1.0.0*:
```ShellSession
calicoctl.sh status
```
* Show the configured network subnet for containers
```ShellSession
calicoctl.sh get ippool -o wide
```
or for versions prior to *v1.0.0*:
```ShellSession
calicoctl.sh pool show
```
* Show the workloads (ip addresses of containers and their location)
```ShellSession
@ -50,12 +32,6 @@ and
calicoctl.sh get hostEndpoint -o wide
```
or for versions prior *v1.0.0*:
```ShellSession
calicoctl.sh endpoint show --detail
```
## Configuration
### Optional : Define datastore type

6
roles/download/defaults/main.yml
View file

@ -361,18 +361,12 @@ calicoctl_binary_checksums:
arm:
v3.16.1: 0
v3.15.2: 0
v3.14.1: 0
v3.13.3: 0
amd64:
v3.16.1: 7c33a841fdf85409c2eee5b287e1212d6c7e82885ec9ffaf690b6019b7b80c1b
v3.15.2: 219ae954501cbe15daeda0ad52e13ec65f99c77548c7d3cbfc4ced5c7149fdf1
v3.14.1: 5fe8a7b00a45cf48879eff42b08dcdb85cf0121f3720ac8cbd06566aaa385667
v3.13.3: 570539d436df51bb349bb1a8c6b200a3a6f20803a9d391aa2c5cf19a70a083d4
arm64:
v3.16.1: d3cc8b721a862f0c50273706bf6d38e47ee9b932b8d90a0f0e51280594a6f242
v3.15.2: 49165f9e4ad55402248b578310fcf68a57363f54e66be04ac24be9714899b4d5
v3.14.1: 326da28cb726988029f70fbf3d4de424a4edd9949fd435fad81f2203c93e4c36
v3.13.3: 0c47acd6d200ba1f8348b389cd7a54771542158fef657afc633a30ddad97e272
etcd_binary_checksum: "{{ etcd_binary_checksums[image_arch] }}"
cni_binary_checksum: "{{ cni_binary_checksums[image_arch] }}"

4
roles/kubernetes-apps/policy_controller/calico/templates/calico-kube-controllers.yml.j2
View file

@ -38,15 +38,13 @@ spec:
requests:
cpu: {{ calico_policy_controller_cpu_requests }}
memory: {{ calico_policy_controller_memory_requests }}
{% if calico_version is version('v3.3.0', '>=') %}
readinessProbe:
exec:
command:
- /usr/bin/check-status
- -r
{% endif %}
env:
{% if calico_datastore == "kdd" and calico_version is version('v3.6.0', '>=') %}
{% if calico_datastore == "kdd" %}
- name: ENABLED_CONTROLLERS
value: node
- name: DATASTORE_TYPE

2
roles/kubernetes-apps/policy_controller/calico/templates/calico-kube-cr.yml.j2
View file

@ -81,7 +81,6 @@ rules:
- get
- create
- update
{% if calico_version is version('v3.14.0', '>=') %}
# KubeControllersConfiguration is where it gets its config
- apiGroups: ["crd.projectcalico.org"]
resources:
@ -96,4 +95,3 @@ rules:
# watch for changes
- watch
{% endif %}
{% endif %}

11
roles/kubernetes/kubeadm/tasks/main.yml
View file

@ -148,17 +148,6 @@
tags:
- kube-proxy
# FIXME(mattymo): Reconcile kubelet kubeconfig filename for both deploy modes
- name: Symlink kubelet kubeconfig for calico/canal
file:
src: "{{ kube_config_dir }}/kubelet.conf"
dest: "{{ kube_config_dir }}/node-kubeconfig.yaml"
state: link
force: yes
when:
- kube_network_plugin in ['calico','canal']
- calico_version is version('v3.3.0', '<')
- name: Extract etcd certs from control plane if using etcd kubeadm mode
include_tasks: kubeadm_etcd_node.yml
when:

12
roles/kubernetes/preinstall/tasks/0020-verify-settings.yml
View file

@ -152,13 +152,13 @@
- name: Ensure minimum calico version
assert:
that: calico_version is version('v3.0.0', '>=')
msg: "calico_version is too low. Minimum version v3.0.0"
that: calico_version is version(calico_min_version_required, '>=')
msg: "calico_version is too low. Minimum version {{ calico_min_version_required }}"
run_once: yes
when:
- kube_network_plugin == 'calico'
- name: "Get current version of calico cluster version"
- name: Get current calico cluster version
shell: "set -o pipefail && {{ bin_dir }}/calicoctl.sh version | grep 'Cluster Version:' | awk '{ print $3}'"
args:
executable: /bin/bash
@ -171,11 +171,11 @@
when:
- kube_network_plugin == 'calico'
- name: "Check that calico version is enough for upgrade"
- name: Check that current calico version is enough for upgrade
assert:
that:
- calico_version_on_server.stdout is version('v2.6.5', '>=')
msg: "Your version of calico is not fresh enough for upgrade. Minimum version v2.6.5"
- calico_version_on_server.stdout is version(calico_min_version_required, '>=')
msg: "Your version of calico is not fresh enough for upgrade. Minimum version {{ calico_min_version_required }}"
when:
- kube_network_plugin == 'calico'
- 'calico_version_on_server.stdout is defined'

5
roles/kubespray-defaults/vars/main.yml
View file

@ -1,2 +1,7 @@
---
# Kubespray constants
kube_proxy_deployed: "{{ 'addon/kube-proxy' not in kubeadm_init_phases_skip }}"
# The lowest version allowed to upgrade from (same as calico_version in the previous branch)
calico_min_version_required: "v3.15.2"

33
roles/network_plugin/calico/tasks/check.yml
View file

@ -36,36 +36,3 @@
msg: "IP in IP and VXLAN mode is mutualy exclusive modes"
when:
- "calico_vxlan_mode in ['Always', 'CrossSubnet']"
- name: "Get current version of calico cluster version"
shell: "set -o pipefail && {{ bin_dir }}/calicoctl.sh version | grep 'Cluster Version:' | awk '{ print $3}'"
args:
executable: /bin/bash
register: calico_version_on_server
async: 10
poll: 3
run_once: yes
changed_when: false
failed_when: false
- name: "Determine if calico upgrade is needed"
block:
- name: "Check that calico version is enough for upgrade"
assert:
that:
- calico_version_on_server.stdout is version('v2.6.5', '>=')
msg: "Your version of calico is not fresh enough for upgrade"
when: calico_upgrade_enabled
- name: "Set upgrade flag when version needs to be updated"
set_fact:
calico_upgrade_needed: True
when:
- calico_version_on_server.stdout is version('v2.6.5', '>=')
- calico_version_on_server.stdout is version('v3.0.0', '<')
when:
- calico_version_on_server.stdout is defined
- calico_version_on_server.stdout
- inventory_hostname == groups['kube-master'][0]
run_once: yes

24
roles/network_plugin/calico/tasks/install.yml
View file

@ -32,7 +32,7 @@
- name: Calico | Write Calico cni config
template:
src: "cni-calico.conflist.j2"
dest: "/etc/cni/net.d/{% if calico_version is version('v3.3.0', '>=') %}calico.conflist.template{% else %}10-calico.conflist{% endif %}"
dest: "/etc/cni/net.d/calico.conflist.template"
owner: kube
register: calico_conflist
notify: reset_calico_cni
@ -135,26 +135,7 @@
loop_control:
label: "{{ item.item.file }}"
- name: Calico | Configure calico network pool (version < v3.3.0)
command:
cmd: "{{ bin_dir }}/calicoctl.sh apply -f -"
stdin: >
{ "kind": "IPPool",
"apiVersion": "projectcalico.org/v3",
"metadata": {
"name": "{{ calico_pool_name }}",
},
"spec": {
"cidr": "{{ calico_pool_cidr | default(kube_pods_subnet) }}",
"ipipMode": "{{ calico_ipip_mode }}",
"vxlanMode": "{{ calico_vxlan_mode }}",
"natOutgoing": {{ nat_outgoing|default(false) and not peer_with_router|default(false) }} }}
when:
- inventory_hostname == groups['kube-master'][0]
- 'calico_conf.stdout == "0"'
- calico_version is version("v3.3.0", "<")
- name: Calico | Configure calico network pool (version >= v3.3.0)
- name: Calico | Configure calico network pool
command:
cmd: "{{ bin_dir }}/calicoctl.sh apply -f -"
stdin: >
@ -172,7 +153,6 @@
when:
- inventory_hostname == groups['kube-master'][0]
- 'calico_conf.stdout == "0"'
- calico_version is version("v3.3.0", ">=")
- name: "Determine nodeToNodeMesh needed state"
set_fact:

7
roles/network_plugin/calico/tasks/main.yml
View file

@ -3,11 +3,4 @@
- import_tasks: pre.yml
- include_tasks: upgrade.yml
when:
- calico_upgrade_enabled
- calico_upgrade_needed
- inventory_hostname in groups['kube-master']
run_once: yes
- include_tasks: install.yml

26
roles/network_plugin/calico/tasks/upgrade.yml
View file

@ -1,26 +0,0 @@
---
- name: "Download calico-upgrade tool (force version)"
get_url:
url: "{{ calico_upgrade_url }}"
dest: "{{ bin_dir }}/calico-upgrade"
mode: 0755
owner: root
group: root
force: yes
environment: "{{ proxy_env }}"
- name: "Create etcdv2 and etcdv3 calicoApiConfig"
template:
src: "{{ item }}-store.yml.j2"
dest: "/etc/calico/{{ item }}.yml"
with_items:
- "etcdv2"
- "etcdv3"
- name: "Tests data migration (dry-run)" # noqa 301 305
shell: "{{ bin_dir }}/calico-upgrade dry-run --output-dir=/tmp --apiconfigv1 /etc/calico/etcdv2.yml --apiconfigv3 /etc/calico/etcdv3.yml"
register: calico_upgrade_test_data
failed_when: '"Successfully" not in calico_upgrade_test_data.stdout'
- name: "If test migration is success continue with calico data real migration" # noqa 301 305
shell: "{{ bin_dir }}/calico-upgrade start --no-prompts --apiconfigv1 /etc/calico/etcdv2.yml --apiconfigv3 /etc/calico/etcdv3.yml --output-dir=/tmp/calico_upgrade"
register: calico_upgrade_migration_data

8
roles/network_plugin/calico/templates/calico-cr.yml.j2
View file

@ -72,20 +72,14 @@ rules:
- globalbgpconfigs
- bgpconfigurations
- ippools
{% if calico_version is version('v3.6.0', '>=') %}
- ipamblocks
{% endif %}
- globalnetworkpolicies
- globalnetworksets
- networkpolicies
{% if calico_version is version('v3.7.0', '>=') %}
- networksets
{% endif %}
- clusterinformations
- hostendpoints
{% if calico_version is version('v3.9.0', '>=') %}
- blockaffinities
{% endif %}
verbs:
- get
- list
@ -116,7 +110,6 @@ rules:
verbs:
- create
- update
{% if calico_version is version('v3.6.0', '>=') %}
# These permissions are required for Calico CNI to perform IPAM allocations.
- apiGroups: ["crd.projectcalico.org"]
resources:
@ -148,4 +141,3 @@ rules:
verbs:
- get
{% endif %}
{% endif %}

49
roles/network_plugin/calico/templates/calico-node.yml.j2
View file

@ -33,9 +33,8 @@ spec:
# Minimize downtime during a rolling upgrade or deletion; tell Kubernetes to do a "force
# deletion": https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods.
terminationGracePeriodSeconds: 0
{% if calico_version is version('v3.4.0', '>=') %}
initContainers:
{% if calico_datastore == "kdd" and calico_version is version('v3.6.0', '>=') %}
{% if calico_datastore == "kdd" %}
# This container performs upgrade from host-local IPAM to calico-ipam.
# It can be deleted if this is a fresh installation, or if you have already
# upgraded to use calico-ipam.
@ -92,31 +91,7 @@ spec:
name: cni-bin-dir
securityContext:
privileged: true
{% endif %}
containers:
{% if calico_version is version('v3.3.0', '>=') and calico_version is version('v3.4.0', '<') %}
- name: install-cni
image: {{ calico_cni_image_repo }}:{{ calico_cni_image_tag }}
command: ["/opt/cni/bin/install"]
env:
# Name of the CNI config file to create.
- name: CNI_CONF_NAME
value: "10-calico.conflist"
# Install CNI binaries
- name: UPDATE_CNI_BINARIES
value: "true"
# The CNI network config to install on each node.
- name: CNI_NETWORK_CONFIG_FILE
value: "/host/etc/cni/net.d/calico.conflist.template"
# Prevents the container from sleeping forever.
- name: SLEEP
value: "false"
volumeMounts:
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
{% endif %}
# Runs calico/node container on each Kubernetes node. This
# container programs network policy and routes on each
# host.
@ -209,10 +184,8 @@ spec:
- name: FELIX_KUBENODEPORTRANGES
value: "{{ kube_apiserver_node_port_range.split('-')[0] }}:{{ kube_apiserver_node_port_range.split('-')[1] }}"
{% endif %}
{% if calico_version is version('v3.8.1', '>=') %}
- name: FELIX_IPTABLESBACKEND
value: "{{ calico_iptables_backend }}"
{% endif %}
- name: FELIX_IPTABLESLOCKTIMEOUTSECS
value: "{{ calico_iptables_lock_timeout_secs }}"
# should be set in etcd before deployment
@ -250,7 +223,7 @@ spec:
value: "{{ calico_felix_prometheusgometricsenabled }}"
- name: FELIX_PROMETHEUSPROCESSMETRICSENABLED
value: "{{ calico_felix_prometheusprocessmetricsenabled }}"
{% if calico_version is version('v3.4.0', '>=') and calico_advertise_cluster_ips|default(false) %}
{% if calico_advertise_cluster_ips|default(false) %}
- name: CALICO_ADVERTISE_CLUSTER_IPS
value: "{{ kube_service_addresses }}"
{% endif %}
@ -265,7 +238,7 @@ spec:
fieldRef:
fieldPath: status.hostIP
{% endif %}
{% if calico_version is version('v3.9.0', '>=') and calico_use_default_route_src_ipaddr|default(false) %}
{% if calico_use_default_route_src_ipaddr|default(false) %}
- name: FELIX_DEVICEROUTESOURCEADDRESS
valueFrom:
fieldRef:
@ -295,30 +268,17 @@ spec:
cpu: {{ calico_node_cpu_requests }}
memory: {{ calico_node_memory_requests }}
livenessProbe:
{% if calico_version is version('v3.8.0', '<') %}
httpGet:
host: 127.0.0.1
path: /liveness
port: 9099
{% else %}
exec:
command:
- /bin/calico-node
- -felix-live
{% if calico_network_backend|default("bird") == "bird" %}
- -bird-live
{% endif %}
{% endif %}
initialDelaySeconds: 5
failureThreshold: 6
readinessProbe:
failureThreshold: 6
{% if calico_version is version('v3.3.0', '<') %}
httpGet:
host: 127.0.0.1
path: /readiness
port: 9099
{% else %}
exec:
command:
- /bin/calico-node
@ -326,7 +286,6 @@ spec:
- -bird-ready
{% endif %}
- -felix-ready
{% endif %}
volumeMounts:
- mountPath: /lib/modules
name: lib-modules
@ -382,7 +341,7 @@ spec:
hostPath:
path: /run/xtables.lock
type: FileOrCreate
{% if calico_datastore == "kdd" and calico_version is version('v3.6.0', '>=') %}
{% if calico_datastore == "kdd" %}
# Mount in the directory for host-local IPAM allocations. This is
# used when upgrading from host-local to calico-ipam, and can be removed
# if not using the upgrade-ipam init container.

16
roles/network_plugin/calico/templates/calico-typha.yml.j2
View file

@ -115,33 +115,17 @@ spec:
# - name: USE_POD_CIDR
# value: "true"
livenessProbe:
{% if calico_version is version('v3.7.0', '<') %}
exec:
command:
- calico-typha
- check
- liveness
{% else %}
httpGet:
path: /liveness
port: 9098
host: localhost
{% endif %}
periodSeconds: 30
initialDelaySeconds: 30
readinessProbe:
{% if calico_version is version('v3.7.0', '<') %}
exec:
command:
- calico-typha
- check
- readiness
{% else %}
httpGet:
path: /readiness
port: 9098
host: localhost
{% endif %}
periodSeconds: 10
{% if typha_secure %}
volumes:

2
roles/network_plugin/calico/templates/cni-calico.conflist.j2
View file

@ -52,7 +52,7 @@
"mtu": {{ calico_mtu }},
{% endif %}
"kubernetes": {
"kubeconfig": "{% if calico_version is version('v3.3.0', '>=') %}__KUBECONFIG_FILEPATH__{% else %}{{ kube_config_dir }}/node-kubeconfig.yaml{% endif %}"
"kubeconfig": "__KUBECONFIG_FILEPATH__"
}
},
{

6
roles/network_plugin/calico/templates/kdd-crds.yml.j2
View file

@ -514,7 +514,6 @@ spec:
storage: true
---
{% if calico_version is version('v3.6.0', '>=') %}
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
@ -740,7 +739,6 @@ spec:
storage: true
---
{% endif %}
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
@ -2693,7 +2691,6 @@ spec:
served: true
storage: true
{% if calico_version is version('v3.7.0', '>=') %}
---
apiVersion: apiextensions.k8s.io/v1
@ -2739,8 +2736,6 @@ spec:
type: object
served: true
storage: true
{% endif %}
{% if calico_version is version('v3.14.0', '>=') %}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
@ -2957,4 +2952,3 @@ spec:
type: object
served: true
storage: true
{% endif %}

2
roles/network_plugin/canal/tasks/main.yml
View file

@ -2,7 +2,7 @@
- name: Canal | Write Canal cni config
template:
src: "cni-canal.conflist.j2"
dest: "/etc/cni/net.d/{% if calico_version is version('v3.3.0', '>=') %}canal.conflist.template{% else %}10-canal.conflist{% endif %}"
dest: "/etc/cni/net.d/canal.conflist.template"
owner: kube
register: canal_conflist
notify: reset_canal_cni

35
roles/network_plugin/canal/templates/canal-node.yaml.j2
View file

@ -53,7 +53,6 @@ spec:
hostPath:
path: /run/xtables.lock
type: FileOrCreate
{% if calico_version is version('v3.4.0', '>=') %}
initContainers:
# This container installs the Calico CNI binaries
# and CNI network config file on each node.
@ -78,28 +77,7 @@ spec:
name: cni-net-dir
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
{% endif %}
containers:
{% if calico_version is version('v3.3.0', '>=') and calico_version is version('v3.4.0', '<') %}
- name: install-cni
image: {{ calico_cni_image_repo }}:{{ calico_cni_image_tag }}
command: ["/opt/cni/bin/install"]
env:
# Name of the CNI config file to create.
- name: CNI_CONF_NAME
value: "10-canal.conflist"
# Install CNI binaries
- name: UPDATE_CNI_BINARIES
value: "true"
# The CNI network config to install on each node.
- name: CNI_NETWORK_CONFIG_FILE
value: "/host/etc/cni/net.d/canal.conflist.template"
volumeMounts:
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
{% endif %}
# Runs the flannel daemon to enable vxlan networking between
# container hosts.
- name: flannel
@ -215,12 +193,6 @@ spec:
fieldPath: spec.nodeName
- name: FELIX_HEALTHENABLED
value: "true"
# Prior to v3.2.1 iptables didn't acquire the lock, so Calico's own implementation of the lock should be used,
# this is not required in later versions https://github.com/projectcalico/calico/issues/2179
{% if calico_version is version('v3.2.1', '<') %}
- name: FELIX_IPTABLESLOCKTIMEOUTSECS
value: "10"
{% endif %}
# Disable IPv6 on Kubernetes.
- name: FELIX_IPV6SUPPORT
value: "false"
@ -255,17 +227,10 @@ spec:
initialDelaySeconds: 5
failureThreshold: 6
readinessProbe:
{% if calico_version is version('v3.3.0', '<')%}
httpGet:
host: 127.0.0.1
path: /readiness
port: 9099
{% else %}
exec:
command:
- /bin/calico-node
- -felix-ready
{% endif %}
periodSeconds: 10
volumeMounts:
- mountPath: /lib/modules

2
roles/network_plugin/canal/templates/cni-canal.conflist.j2
View file

@ -17,7 +17,7 @@
"type": "k8s"
},
"kubernetes": {
"kubeconfig": "{% if calico_version is version('v3.3.0', '>=') %}__KUBECONFIG_FILEPATH__{% else %}{{ kube_config_dir }}/node-kubeconfig.yaml{% endif %}"
"kubeconfig": "__KUBECONFIG_FILEPATH__"
}
}
},