2021-06-01 14:30:27 +00:00
|
|
|
# Local Storage Provisioner
|
2017-11-01 14:25:35 +00:00
|
|
|
|
2019-01-08 20:36:44 +00:00
|
|
|
The [local storage provisioner](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume)
|
|
|
|
is NOT a dynamic storage provisioner as you would
|
2017-11-01 14:25:35 +00:00
|
|
|
expect from a cloud provider. Instead, it simply creates PersistentVolumes for
|
2019-01-08 20:36:44 +00:00
|
|
|
all mounts under the host_dir of the specified storage class.
|
2019-01-08 23:32:39 +00:00
|
|
|
These storage classes are specified in the `local_volume_provisioner_storage_classes` nested dictionary.
|
|
|
|
Example:
|
2019-01-08 20:36:44 +00:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
local_volume_provisioner_storage_classes:
|
2019-01-08 23:32:39 +00:00
|
|
|
local-storage:
|
|
|
|
host_dir: /mnt/disks
|
|
|
|
mount_dir: /mnt/disks
|
|
|
|
fast-disks:
|
|
|
|
host_dir: /mnt/fast-disks
|
|
|
|
mount_dir: /mnt/fast-disks
|
|
|
|
block_cleaner_command:
|
|
|
|
- "/scripts/shred.sh"
|
|
|
|
- "2"
|
|
|
|
volume_mode: Filesystem
|
|
|
|
fs_type: ext4
|
2019-01-08 20:36:44 +00:00
|
|
|
```
|
|
|
|
|
2019-01-08 23:32:39 +00:00
|
|
|
For each key in `local_volume_provisioner_storage_classes` a storageClass with the
|
|
|
|
same name is created. The subkeys of each storage class are converted to camelCase and added
|
2019-01-08 20:36:44 +00:00
|
|
|
as attributes to the storageClass.
|
|
|
|
The result of the above example is:
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
data:
|
|
|
|
storageClassMap: |
|
|
|
|
local-storage:
|
2019-01-08 23:32:39 +00:00
|
|
|
hostDir: /mnt/disks
|
|
|
|
mountDir: /mnt/disks
|
2019-01-08 20:36:44 +00:00
|
|
|
fast-disks:
|
2019-01-08 23:32:39 +00:00
|
|
|
hostDir: /mnt/fast-disks
|
|
|
|
mountDir: /mnt/fast-disks
|
|
|
|
blockCleanerCommand:
|
|
|
|
- "/scripts/shred.sh"
|
|
|
|
- "2"
|
|
|
|
volumeMode: Filesystem
|
|
|
|
fsType: ext4
|
2019-01-08 20:36:44 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
The default StorageClass is local-storage on /mnt/disks,
|
|
|
|
the rest of this doc will use that path as an example.
|
2017-11-01 14:25:35 +00:00
|
|
|
|
2021-06-01 14:30:27 +00:00
|
|
|
## Examples to create local storage volumes
|
2017-11-01 14:25:35 +00:00
|
|
|
|
2020-12-30 13:07:49 +00:00
|
|
|
1. tmpfs method:
|
2017-11-01 14:25:35 +00:00
|
|
|
|
2018-02-15 01:55:43 +00:00
|
|
|
``` bash
|
|
|
|
for vol in vol1 vol2 vol3; do
|
|
|
|
mkdir /mnt/disks/$vol
|
|
|
|
mount -t tmpfs -o size=5G $vol /mnt/disks/$vol
|
|
|
|
done
|
|
|
|
```
|
2017-11-01 14:25:35 +00:00
|
|
|
|
|
|
|
The tmpfs method is not recommended for production because the mount is not
|
|
|
|
persistent and data will be deleted on reboot.
|
|
|
|
|
2020-12-30 13:07:49 +00:00
|
|
|
1. Mount physical disks
|
2017-11-01 14:25:35 +00:00
|
|
|
|
2018-02-15 01:55:43 +00:00
|
|
|
``` bash
|
|
|
|
mkdir /mnt/disks/ssd1
|
|
|
|
mount /dev/vdb1 /mnt/disks/ssd1
|
|
|
|
```
|
2017-11-01 14:25:35 +00:00
|
|
|
|
|
|
|
Physical disks are recommended for production environments because it offers
|
|
|
|
complete isolation in terms of I/O and capacity.
|
|
|
|
|
2020-12-30 13:07:49 +00:00
|
|
|
1. Mount unpartitioned physical devices
|
2019-07-16 12:27:26 +00:00
|
|
|
|
|
|
|
``` bash
|
|
|
|
for disk in /dev/sdc /dev/sdd /dev/sde; do
|
|
|
|
ln -s $disk /mnt/disks
|
|
|
|
done
|
|
|
|
```
|
|
|
|
|
2021-05-31 12:46:26 +00:00
|
|
|
This saves time of precreating filesystems. Note that your storageclass must have
|
2019-07-16 12:27:26 +00:00
|
|
|
volume_mode set to "Filesystem" and fs_type defined. If either is not set, the
|
|
|
|
disk will be added as a raw block device.
|
|
|
|
|
2020-12-30 13:07:49 +00:00
|
|
|
1. File-backed sparsefile method
|
2017-11-01 14:25:35 +00:00
|
|
|
|
2018-02-15 01:55:43 +00:00
|
|
|
``` bash
|
|
|
|
truncate /mnt/disks/disk5 --size 2G
|
|
|
|
mkfs.ext4 /mnt/disks/disk5
|
|
|
|
mkdir /mnt/disks/vol5
|
|
|
|
mount /mnt/disks/disk5 /mnt/disks/vol5
|
|
|
|
```
|
2017-11-01 14:25:35 +00:00
|
|
|
|
|
|
|
If you have a development environment and only one disk, this is the best way
|
|
|
|
to limit the quota of persistent volumes.
|
|
|
|
|
2020-12-30 13:07:49 +00:00
|
|
|
1. Simple directories
|
2018-02-15 01:55:43 +00:00
|
|
|
|
2018-08-10 14:14:34 +00:00
|
|
|
In a development environment using `mount --bind` works also, but there is no capacity
|
2017-11-01 14:25:35 +00:00
|
|
|
management.
|
|
|
|
|
2020-12-30 13:07:49 +00:00
|
|
|
1. Block volumeMode PVs
|
2018-08-10 14:14:34 +00:00
|
|
|
|
|
|
|
Create a symbolic link under discovery directory to the block device on the node. To use
|
2019-07-16 12:27:26 +00:00
|
|
|
raw block devices in pods, volume_type should be set to "Block".
|
2018-08-10 14:14:34 +00:00
|
|
|
|
2021-06-01 14:30:27 +00:00
|
|
|
## Usage notes
|
2017-11-01 14:25:35 +00:00
|
|
|
|
2018-08-10 14:14:34 +00:00
|
|
|
Beta PV.NodeAffinity field is used by default. If running against an older K8s
|
|
|
|
version, the useAlphaAPI flag must be set in the configMap.
|
|
|
|
|
2017-11-01 14:25:35 +00:00
|
|
|
The volume provisioner cannot calculate volume sizes correctly, so you should
|
|
|
|
delete the daemonset pod on the relevant host after creating volumes. The pod
|
|
|
|
will be recreated and read the size correctly.
|
|
|
|
|
|
|
|
Make sure to make any mounts persist via /etc/fstab or with systemd mounts (for
|
2020-08-28 09:28:53 +00:00
|
|
|
Flatcar Container Linux). Pods with persistent volume claims will not be
|
2017-11-01 14:25:35 +00:00
|
|
|
able to start if the mounts become unavailable.
|
|
|
|
|
2021-06-01 14:30:27 +00:00
|
|
|
## Further reading
|
2017-11-01 14:25:35 +00:00
|
|
|
|
2018-02-15 01:55:43 +00:00
|
|
|
Refer to the upstream docs here: <https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume>
|