AWS EBS CSI implementation (#5549)

* AWS EBS CSI implementation

* Fixing image repos

* Add OWNERS file

* Fix expressions

* Add csi-driver tag

* Add AWS EBS prefix to variables

* Add AWS EBS CSI Driver documentation
This commit is contained in:
Ali Sanhaji 2020-03-25 21:10:25 +01:00 committed by GitHub
parent 63fa406c3c
commit a8a05a21a4
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
19 changed files with 665 additions and 41 deletions

87
docs/aws-ebs-csi.md Normal file
View file

@ -0,0 +1,87 @@
# AWS EBS CSI Driver
AWS EBS CSI driver allows you to provision EBS volumes for pods in EC2 instances. The old in-tree AWS cloud provider is deprecated and will be removed in future versions of Kubernetes. So transitioning to the CSI driver is advised.
To enable AWS EBS CSI driver, uncomment the `aws_ebs_csi_enabled` option in `group_vars/all/aws.yml` and set it to `true`.
To set the number of replicas for the AWS CSI controller, you can change `aws_ebs_csi_controller_replicas` option in `group_vars/all/aws.yml`.
Make sure to add a role, for your EC2 instances hosting Kubernetes, that allows it to do the actions necessary to request a volume and attach it: [AWS CSI Policy](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/example-iam-policy.json)
If you want to deploy the AWS EBS storage class used with the CSI Driver, you should set `persistent_volumes_enabled` in `group_vars/k8s-cluster/k8s-cluster.yml` to `true`.
You can now run the kubespray playbook (cluster.yml) to deploy Kubernetes over AWS EC2 with EBS CSI Driver enabled.
## Usage example
To check if AWS EBS CSI Driver is deployed properly, check that the ebs-csi pods are running:
```ShellSession
$ kubectl -n kube-system get pods | grep ebs
ebs-csi-controller-85d86bccc5-8gtq5 4/4 Running 4 40s
ebs-csi-node-n4b99 3/3 Running 3 40s
```
Check the associated storage class (if you enabled persistent_volumes):
```ShellSession
$ kubectl get storageclass
NAME PROVISIONER AGE
ebs-sc ebs.csi.aws.com 45s
```
You can run a PVC and an example Pod using this file `ebs-pod.yml`:
```yml
--
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ebs-claim
spec:
accessModes:
- ReadWriteOnce
storageClassName: ebs-sc
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
containers:
- name: app
image: centos
command: ["/bin/sh"]
args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"]
volumeMounts:
- name: persistent-storage
mountPath: /data
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: ebs-claim
```
Apply this conf to your cluster: ```kubectl apply -f ebs-pod.yml```
You should see the PVC provisioned and bound:
```ShellSession
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
ebs-claim Bound pvc-0034cb9e-1ddd-4b3f-bb9e-0b5edbf5194c 1Gi RWO ebs-sc 50s
```
And the volume mounted to the example Pod (wait until the Pod is Running):
```ShellSession
$ kubectl exec -it app -- df -h | grep data
/dev/nvme1n1 1014M 34M 981M 4% /data
```
## More info
For further information about the AWS EBS CSI Driver, you can refer to this page: [AWS EBS Driver](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/).

View file

@ -0,0 +1,8 @@
## To use AWS EBS CSI Driver to provision volumes, uncomment the first value
## and configure the parameters below
# aws_ebs_csi_enabled: true
# aws_ebs_csi_enable_volume_scheduling: true
# aws_ebs_csi_enable_volume_snapshot: false
# aws_ebs_csi_enable_volume_resizing: false
# aws_ebs_csi_controller_replicas: 1
# aws_ebs_csi_plugin_image_tag: latest

View file

@ -255,7 +255,7 @@ podsecuritypolicy_enabled: false
## See https://github.com/kubernetes-sigs/kubespray/issues/2141 ## See https://github.com/kubernetes-sigs/kubespray/issues/2141
## Set this variable to true to get rid of this issue ## Set this variable to true to get rid of this issue
volume_cross_zone_attachment: false volume_cross_zone_attachment: false
# Add Persistent Volumes Storage Class for corresponding cloud provider ( OpenStack is only supported now ) # Add Persistent Volumes Storage Class for corresponding cloud provider (supported: in-tree OpenStack, Cinder CSI, AWS EBS CSI)
persistent_volumes_enabled: false persistent_volumes_enabled: false
## Container Engine Acceleration ## Container Engine Acceleration

View file

@ -494,18 +494,24 @@ addon_resizer_version: "1.8.8"
addon_resizer_image_repo: "{{ kube_image_repo }}/addon-resizer" addon_resizer_image_repo: "{{ kube_image_repo }}/addon-resizer"
addon_resizer_image_tag: "{{ addon_resizer_version }}" addon_resizer_image_tag: "{{ addon_resizer_version }}"
cinder_csi_attacher_image_repo: "{{ quay_image_repo }}/k8scsi/csi-attacher" csi_attacher_image_repo: "{{ quay_image_repo }}/k8scsi/csi-attacher"
cinder_csi_attacher_image_tag: "v1.2.1" csi_attacher_image_tag: "v1.2.1"
cinder_csi_provisioner_image_repo: "{{ quay_image_repo }}/k8scsi/csi-provisioner" csi_provisioner_image_repo: "{{ quay_image_repo }}/k8scsi/csi-provisioner"
cinder_csi_provisioner_image_tag: "v1.3.0" csi_provisioner_image_tag: "v1.3.0"
cinder_csi_snapshotter_image_repo: "{{ quay_image_repo }}/k8scsi/csi-snapshotter" csi_snapshotter_image_repo: "{{ quay_image_repo }}/k8scsi/csi-snapshotter"
cinder_csi_snapshotter_image_tag: "v1.2.0" csi_snapshotter_image_tag: "v1.2.0"
cinder_csi_resizer_image_repo: "{{ quay_image_repo }}/k8scsi/csi-resizer" csi_resizer_image_repo: "{{ quay_image_repo }}/k8scsi/csi-resizer"
cinder_csi_resizer_image_tag: "v0.2.0" csi_resizer_image_tag: "v0.2.0"
csi_node_driver_registrar_image_repo: "{{ quay_image_repo }}/k8scsi/csi-node-driver-registrar"
csi_node_driver_registrar_image_tag: "v1.1.0"
csi_livenessprobe_image_repo: "{{ quay_image_repo }}/k8scsi/livenessprobe"
csi_livenessprobe_image_tag: "v1.1.0"
cinder_csi_plugin_image_repo: "{{ docker_image_repo }}/k8scloudprovider/cinder-csi-plugin" cinder_csi_plugin_image_repo: "{{ docker_image_repo }}/k8scloudprovider/cinder-csi-plugin"
cinder_csi_plugin_image_tag: "latest" cinder_csi_plugin_image_tag: "latest"
cinder_csi_node_driver_registrar_image_repo: "{{ quay_image_repo }}/k8scsi/csi-node-driver-registrar"
cinder_csi_node_driver_registrar_image_tag: "v1.1.0" aws_ebs_csi_plugin_image_repo: "{{ docker_image_repo }}/amazon/aws-ebs-csi-driver"
aws_ebs_csi_plugin_image_tag: "latest"
dashboard_image_repo: "{{ gcr_image_repo }}/google_containers/kubernetes-dashboard-{{ image_arch }}" dashboard_image_repo: "{{ gcr_image_repo }}/google_containers/kubernetes-dashboard-{{ image_arch }}"
dashboard_image_tag: "v1.10.1" dashboard_image_tag: "v1.10.1"
@ -1011,39 +1017,48 @@ downloads:
groups: groups:
- kube-node - kube-node
cinder_csi_attacher: csi_attacher:
enabled: "{{ cinder_csi_enabled }}" enabled: "{{ cinder_csi_enabled or aws_ebs_csi_enabled }}"
container: true container: true
repo: "{{ cinder_csi_attacher_image_repo }}" repo: "{{ csi_attacher_image_repo }}"
tag: "{{ cinder_csi_attacher_image_tag }}" tag: "{{ csi_attacher_image_tag }}"
sha256: "{{ cinder_csi_attacher_digest_checksum|default(None) }}" sha256: "{{ csi_attacher_digest_checksum|default(None) }}"
groups: groups:
- kube-node - kube-node
cinder_csi_provisioner: csi_provisioner:
enabled: "{{ cinder_csi_enabled }}" enabled: "{{ cinder_csi_enabled or aws_ebs_csi_enabled }}"
container: true container: true
repo: "{{ cinder_csi_provisioner_image_repo }}" repo: "{{ csi_provisioner_image_repo }}"
tag: "{{ cinder_csi_provisioner_image_tag }}" tag: "{{ csi_provisioner_image_tag }}"
sha256: "{{ cinder_csi_provisioner_digest_checksum|default(None) }}" sha256: "{{ csi_provisioner_digest_checksum|default(None) }}"
groups: groups:
- kube-node - kube-node
cinder_csi_snapshotter: csi_snapshotter:
enabled: "{{ cinder_csi_enabled }}" enabled: "{{ cinder_csi_enabled or aws_ebs_csi_enabled }}"
container: true container: true
repo: "{{ cinder_csi_snapshotter_image_repo }}" repo: "{{ csi_snapshotter_image_repo }}"
tag: "{{ cinder_csi_snapshotter_image_tag }}" tag: "{{ csi_snapshotter_image_tag }}"
sha256: "{{ cinder_csi_snapshotter_digest_checksum|default(None) }}" sha256: "{{ csi_snapshotter_digest_checksum|default(None) }}"
groups: groups:
- kube-node - kube-node
cinder_csi_resizer: csi_resizer:
enabled: "{{ cinder_csi_enabled }}" enabled: "{{ cinder_csi_enabled or aws_ebs_csi_enabled }}"
container: true container: true
repo: "{{ cinder_csi_resizer_image_repo }}" repo: "{{ csi_resizer_image_repo }}"
tag: "{{ cinder_csi_resizer_image_tag }}" tag: "{{ csi_resizer_image_tag }}"
sha256: "{{ cinder_csi_resizer_digest_checksum|default(None) }}" sha256: "{{ csi_resizer_digest_checksum|default(None) }}"
groups:
- kube-node
csi_node_driver_registrar:
enabled: "{{ cinder_csi_enabled or aws_ebs_csi_enabled }}"
container: true
repo: "{{ csi_node_driver_registrar_image_repo }}"
tag: "{{ csi_node_driver_registrar_image_tag }}"
sha256: "{{ csi_node_driver_registrar_digest_checksum|default(None) }}"
groups: groups:
- kube-node - kube-node
@ -1056,12 +1071,12 @@ downloads:
groups: groups:
- kube-node - kube-node
cinder_csi_node_driver_registrar: aws_ebs_csi_plugin:
enabled: "{{ cinder_csi_enabled }}" enabled: "{{ aws_ebs_csi_enabled }}"
container: true container: true
repo: "{{ cinder_csi_node_driver_registrar_image_repo }}" repo: "{{ aws_ebs_csi_plugin_image_repo }}"
tag: "{{ cinder_csi_node_driver_registrar_image_tag }}" tag: "{{ aws_ebs_csi_plugin_image_tag }}"
sha256: "{{ cinder_csi_node_driver_registrar_digest_checksum|default(None) }}" sha256: "{{ aws_ebs_csi_plugin_digest_checksum|default(None) }}"
groups: groups:
- kube-node - kube-node

View file

@ -0,0 +1,6 @@
---
aws_ebs_csi_enable_volume_scheduling: true
aws_ebs_csi_enable_volume_snapshot: false
aws_ebs_csi_enable_volume_resizing: false
aws_ebs_csi_controller_replicas: 1
aws_ebs_csi_plugin_image_tag: latest

View file

@ -0,0 +1,27 @@
---
- name: AWS CSI Driver | Generate Manifests
template:
src: "{{ item.file }}.j2"
dest: "{{ kube_config_dir }}/{{ item.file }}"
with_items:
- {name: aws-ebs-csi-driver, file: aws-ebs-csi-driver.yml}
- {name: aws-ebs-csi-controllerservice, file: aws-ebs-csi-controllerservice-rbac.yml}
- {name: aws-ebs-csi-controllerservice, file: aws-ebs-csi-controllerservice.yml}
- {name: aws-ebs-csi-nodeservice, file: aws-ebs-csi-nodeservice.yml}
register: aws_csi_manifests
when: inventory_hostname == groups['kube-master'][0]
tags: aws-ebs-csi-driver
- name: AWS CSI Driver | Apply Manifests
kube:
kubectl: "{{ bin_dir }}/kubectl"
filename: "{{ kube_config_dir }}/{{ item.item.file }}"
state: "latest"
with_items:
- "{{ aws_csi_manifests.results }}"
when:
- inventory_hostname == groups['kube-master'][0]
- not item is skipped
loop_control:
label: "{{ item.item.file }}"
tags: aws-ebs-csi-driver

View file

@ -0,0 +1,179 @@
# Controller Service
apiVersion: v1
kind: ServiceAccount
metadata:
name: ebs-csi-controller-sa
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: ebs-external-provisioner-role
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "watch", "list", "delete", "update", "create"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: ebs-csi-provisioner-binding
subjects:
- kind: ServiceAccount
name: ebs-csi-controller-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: ebs-external-provisioner-role
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: ebs-external-attacher-role
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: ["csi.storage.k8s.io"]
resources: ["csinodeinfos"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: ebs-csi-attacher-binding
subjects:
- kind: ServiceAccount
name: ebs-csi-controller-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: ebs-external-attacher-role
apiGroup: rbac.authorization.k8s.io
{% if aws_ebs_csi_enable_volume_snapshot %}
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: ebs-external-snapshotter-role
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create", "list", "watch", "delete"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: ebs-csi-snapshotter-binding
subjects:
- kind: ServiceAccount
name: ebs-csi-controller-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: ebs-external-snapshotter-role
apiGroup: rbac.authorization.k8s.io
{% endif %}
{% if aws_ebs_csi_enable_volume_resizing %}
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: ebs-external-resizer-role
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims/status"]
verbs: ["update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["list", "watch", "create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: ebs-csi-resizer-binding
subjects:
- kind: ServiceAccount
name: ebs-csi-controller-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: ebs-external-resizer-role
apiGroup: rbac.authorization.k8s.io
{% endif %}

View file

@ -0,0 +1,127 @@
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: ebs-csi-controller
namespace: kube-system
spec:
replicas: {{ aws_ebs_csi_controller_replicas }}
selector:
matchLabels:
app: ebs-csi-controller
app.kubernetes.io/name: aws-ebs-csi-driver
template:
metadata:
labels:
app: ebs-csi-controller
app.kubernetes.io/name: aws-ebs-csi-driver
spec:
nodeSelector:
beta.kubernetes.io/os: linux
serviceAccount: ebs-csi-controller-sa
priorityClassName: system-cluster-critical
tolerations:
- key: CriticalAddonsOnly
operator: Exists
containers:
- name: ebs-plugin
image: {{ aws_ebs_csi_plugin_image_repo }}:{{ aws_ebs_csi_plugin_image_tag }}
args:
- --endpoint=$(CSI_ENDPOINT)
- --logtostderr
- --v=5
env:
- name: CSI_ENDPOINT
value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: aws-secret
key: key_id
optional: true
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: aws-secret
key: access_key
optional: true
volumeMounts:
- name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/
ports:
- name: healthz
containerPort: 9808
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 10
timeoutSeconds: 3
periodSeconds: 10
failureThreshold: 5
- name: csi-provisioner
image: {{ csi_provisioner_image_repo }}:{{ csi_provisioner_image_tag }}
args:
- --csi-address=$(ADDRESS)
- --v=5
{% if aws_ebs_csi_enable_volume_scheduling %}
- --feature-gates=Topology=true
{% endif %}
- --enable-leader-election
- --leader-election-type=leases
env:
- name: ADDRESS
value: /var/lib/csi/sockets/pluginproxy/csi.sock
volumeMounts:
- name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/
- name: csi-attacher
image: {{ csi_attacher_image_repo }}:{{ csi_attacher_image_tag }}
args:
- --csi-address=$(ADDRESS)
- --v=5
env:
- name: ADDRESS
value: /var/lib/csi/sockets/pluginproxy/csi.sock
volumeMounts:
- name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/
{% if aws_ebs_csi_enable_volume_snapshot %}
- name: csi-snapshotter
image: {{ csi_snapshotter_image_repo }}:{{ csi_snapshotter_image_tag }}
args:
- --csi-address=$(ADDRESS)
- --connection-timeout=15s
env:
- name: ADDRESS
value: /var/lib/csi/sockets/pluginproxy/csi.sock
volumeMounts:
- name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/
{% endif %}
{% if aws_ebs_csi_enable_volume_resizing %}
- name: csi-resizer
image: {{ csi_resizer_image_repo }}:{{ csi_resizer_image_tag }}
imagePullPolicy: Always
args:
- --csi-address=$(ADDRESS)
- --v=5
env:
- name: ADDRESS
value: /var/lib/csi/sockets/pluginproxy/csi.sock
volumeMounts:
- name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/
{% endif %}
- name: liveness-probe
image: {{ csi_livenessprobe_image_repo }}:{{ csi_livenessprobe_image_tag }}
args:
- --csi-address=/csi/csi.sock
volumeMounts:
- name: socket-dir
mountPath: /csi
volumes:
- name: socket-dir
emptyDir: {}

View file

@ -0,0 +1,8 @@
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: ebs.csi.aws.com
spec:
attachRequired: true
podInfoOnMount: false

View file

@ -0,0 +1,101 @@
---
# Node Service
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: ebs-csi-node
namespace: kube-system
spec:
selector:
matchLabels:
app: ebs-csi-node
app.kubernetes.io/name: aws-ebs-csi-driver
template:
metadata:
labels:
app: ebs-csi-node
app.kubernetes.io/name: aws-ebs-csi-driver
spec:
nodeSelector:
beta.kubernetes.io/os: linux
hostNetwork: true
priorityClassName: system-node-critical
tolerations:
- key: CriticalAddonsOnly
operator: Exists
containers:
- name: ebs-plugin
securityContext:
privileged: true
image: {{ aws_ebs_csi_plugin_image_repo }}:{{ aws_ebs_csi_plugin_image_tag }}
args:
- --endpoint=$(CSI_ENDPOINT)
- --logtostderr
- --v=5
env:
- name: CSI_ENDPOINT
value: unix:/csi/csi.sock
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet
mountPropagation: "Bidirectional"
- name: plugin-dir
mountPath: /csi
- name: device-dir
mountPath: /dev
ports:
- name: healthz
containerPort: 9808
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 10
timeoutSeconds: 3
periodSeconds: 10
failureThreshold: 5
- name: node-driver-registrar
image: {{ csi_node_driver_registrar_image_repo }}:{{ csi_node_driver_registrar_image_tag }}
args:
- --csi-address=$(ADDRESS)
- --kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)
- --v=5
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "rm -rf /registration/ebs.csi.aws.com-reg.sock /csi/csi.sock"]
env:
- name: ADDRESS
value: /csi/csi.sock
- name: DRIVER_REG_SOCK_PATH
value: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
volumeMounts:
- name: plugin-dir
mountPath: /csi
- name: registration-dir
mountPath: /registration
- name: liveness-probe
image: {{ csi_livenessprobe_image_repo }}:{{ csi_livenessprobe_image_tag }}
args:
- --csi-address=/csi/csi.sock
volumeMounts:
- name: plugin-dir
mountPath: /csi
volumes:
- name: kubelet-dir
hostPath:
path: /var/lib/kubelet
type: Directory
- name: plugin-dir
hostPath:
path: /var/lib/kubelet/plugins/ebs.csi.aws.com/
type: DirectoryOrCreate
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry/
type: Directory
- name: device-dir
hostPath:
path: /dev
type: Directory

View file

@ -20,7 +20,7 @@ spec:
serviceAccount: csi-cinder-controller-sa serviceAccount: csi-cinder-controller-sa
containers: containers:
- name: csi-attacher - name: csi-attacher
image: {{ cinder_csi_attacher_image_repo }}:{{ cinder_csi_attacher_image_tag }} image: {{ csi_attacher_image_repo }}:{{ csi_attacher_image_tag }}
args: args:
- "--v=5" - "--v=5"
- "--csi-address=$(ADDRESS)" - "--csi-address=$(ADDRESS)"
@ -37,7 +37,7 @@ spec:
- name: socket-dir - name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/ mountPath: /var/lib/csi/sockets/pluginproxy/
- name: csi-provisioner - name: csi-provisioner
image: {{ cinder_csi_provisioner_image_repo }}:{{ cinder_csi_provisioner_image_tag }} image: {{ csi_provisioner_image_repo }}:{{ csi_provisioner_image_tag }}
args: args:
- "--csi-address=$(ADDRESS)" - "--csi-address=$(ADDRESS)"
{% if cinder_topology is defined and cinder_topology %} {% if cinder_topology is defined and cinder_topology %}
@ -56,7 +56,7 @@ spec:
- name: socket-dir - name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/ mountPath: /var/lib/csi/sockets/pluginproxy/
- name: csi-snapshotter - name: csi-snapshotter
image: {{ cinder_csi_snapshotter_image_repo }}:{{ cinder_csi_snapshotter_image_tag }} image: {{ csi_snapshotter_image_repo }}:{{ csi_snapshotter_image_tag }}
args: args:
- "--csi-address=$(ADDRESS)" - "--csi-address=$(ADDRESS)"
{% if cinder_csi_controller_replicas is defined and cinder_csi_controller_replicas > 1 %} {% if cinder_csi_controller_replicas is defined and cinder_csi_controller_replicas > 1 %}
@ -71,7 +71,7 @@ spec:
- mountPath: /var/lib/csi/sockets/pluginproxy/ - mountPath: /var/lib/csi/sockets/pluginproxy/
name: socket-dir name: socket-dir
- name: csi-resizer - name: csi-resizer
image: {{ cinder_csi_resizer_image_repo }}:{{ cinder_csi_resizer_image_tag }} image: {{ csi_resizer_image_repo }}:{{ csi_resizer_image_tag }}
args: args:
- "--csi-address=$(ADDRESS)" - "--csi-address=$(ADDRESS)"
{% if cinder_csi_controller_replicas is defined and cinder_csi_controller_replicas > 1 %} {% if cinder_csi_controller_replicas is defined and cinder_csi_controller_replicas > 1 %}

View file

@ -19,7 +19,7 @@ spec:
hostNetwork: true hostNetwork: true
containers: containers:
- name: node-driver-registrar - name: node-driver-registrar
image: {{ cinder_csi_node_driver_registrar_image_repo }}:{{ cinder_csi_node_driver_registrar_image_tag }} image: {{ csi_node_driver_registrar_image_repo }}:{{ csi_node_driver_registrar_image_tag }}
args: args:
- "--csi-address=$(ADDRESS)" - "--csi-address=$(ADDRESS)"
- "--kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)" - "--kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)"

View file

@ -37,6 +37,14 @@ dependencies:
- cinder-csi-driver - cinder-csi-driver
- csi-driver - csi-driver
- role: kubernetes-apps/csi_driver/aws_ebs
when:
- aws_ebs_csi_enabled
tags:
- apps
- aws-ebs-csi-driver
- csi-driver
- role: kubernetes-apps/persistent_volumes - role: kubernetes-apps/persistent_volumes
when: when:
- persistent_volumes_enabled - persistent_volumes_enabled

View file

@ -0,0 +1,5 @@
# See the OWNERS docs at https://go.k8s.io/owners
approvers:
- alijahnas
reviewers:

View file

@ -0,0 +1,8 @@
---
# To restrict which AZ the volume should be provisioned in
# set this value to true and set the list of relevant AZs
# For it to work, the flag aws_ebs_csi_enable_volume_scheduling
# in AWS EBS Driver must be true
restrict_az_provisioning: false
aws_ebs_availability_zones:
- eu-west-3c

View file

@ -0,0 +1,19 @@
---
- name: Kubernetes Persistent Volumes | Copy AWS EBS CSI Storage Class template
template:
src: "aws-ebs-csi-storage-class.yml.j2"
dest: "{{ kube_config_dir }}/aws-ebs-csi-storage-class.yml"
register: manifests
when:
- inventory_hostname == groups['kube-master'][0]
- name: Kubernetes Persistent Volumes | Add AWS EBS CSI Storage Class
kube:
name: aws-ebs-csi
kubectl: "{{ bin_dir }}/kubectl"
resource: StorageClass
filename: "{{ kube_config_dir }}/aws-ebs-csi-storage-class.yml"
state: "latest"
when:
- inventory_hostname == groups['kube-master'][0]
- manifests.changed

View file

@ -0,0 +1,18 @@
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: ebs-sc
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
csi.storage.k8s.io/fstype: xfs
type: gp2
{% if restrict_az_provisioning %}
allowedTopologies:
- matchLabelExpressions:
- key: topology.ebs.csi.aws.com/zone
values:
{% for value in aws_ebs_availability_zones %}
- {{ value }}
{% endfor %}
{% endif %}

View file

@ -13,3 +13,10 @@ dependencies:
tags: tags:
- persistent_volumes_cinder_csi - persistent_volumes_cinder_csi
- cinder-csi-driver - cinder-csi-driver
- role: kubernetes-apps/persistent_volumes/aws-ebs-csi
when:
- aws_ebs_csi_enabled
tags:
- persistent_volumes_aws_ebs_csi
- aws-ebs-csi-driver

View file

@ -304,6 +304,7 @@ enable_network_policy: true
local_volume_provisioner_enabled: "{{ local_volumes_enabled | default('false') }}" local_volume_provisioner_enabled: "{{ local_volumes_enabled | default('false') }}"
local_volume_provisioner_directory_mode: 0700 local_volume_provisioner_directory_mode: 0700
cinder_csi_enabled: false cinder_csi_enabled: false
aws_ebs_csi_enabled: false
persistent_volumes_enabled: false persistent_volumes_enabled: false
cephfs_provisioner_enabled: false cephfs_provisioner_enabled: false
rbd_provisioner_enabled: false rbd_provisioner_enabled: false