Understand Kubernetes storage basics

Persistent volumes and persistent volume claims

Before you get started with provisioning storage, it is important to understand the Kubernetes concepts of a persistent volume and a persistent volume claim and how they work together in a cluster.

The following image shows the storage components in a cluster.

Cluster
By default, every cluster is set up with a plug-in to provision file storage. We can choose to install other add-ons, such as the one for block storage. To use storage in a cluster, we must create a persistent volume claim, a persistent volume and a physical storage instance. When you delete the cluster, we have the option to delete related storage instances.
App
To read from and write to your storage instance, we must mount the persistent volume claim (PVC) to the app. Different storage types have different read-write rules. For example, we can mount multiple pods to the same PVC for file storage. Block storage comes with a RWO (ReadWriteOnce) access mode so that we can mount the storage to one pod only.
Persistent volume claim (PVC)
A PVC is the request to provision persistent storage with a specific type and configuration. To specify the persistent storage flavor that we want, you use Kubernetes storage classes. The cluster admin can define storage classes, or we can choose from one of the predefined storage classes in Red Hat OpenShift on IBM Cloud. When you create a PVC, the request is sent to the IBM Cloud® storage provider. Depending on the configuration that is defined in the storage class, the physical storage device is ordered and provisioned into your IBM Cloud infrastructure account. If the requested configuration does not exist, the storage is not created.
Persistent volume (PV)
A PV is a virtual storage instance that is added as a volume to the cluster. The PV points to a physical storage device in your IBM Cloud infrastructure account and abstracts the API that is used to communicate with the storage device. To mount a PV to an app, we must have a matching PVC. Mounted PVs appear as a folder inside the container's file system.
Physical storage
A physical storage instance that we can use to persist your data. Examples of physical storage in IBM Cloud include File Storage, Block Storage, Object Storage, and local worker node storage that we can use as SDS storage with Portworx. IBM Cloud provides high availability for physical storage instances. However, data that is stored on a physical storage instance is not backed up automatically. Depending on the type of storage that you use, different methods exist to set up backup and restore solutions.

For more information about how to create and use PVCs, PVs, and the physical storage device, see:

Dynamic provisioning

Use dynamic provisioning if we want to give developers the freedom to provision storage when they need it.

How does it work?

Dynamic provisioning is a feature that is native to Kubernetes and that allows a cluster developer to order storage with a pre-defined type and configuration without knowing all the details about how to provision the physical storage device. To abstract the details for the specific storage type, the cluster admin must create storage classes that the developer can use, or use the storage classes that are provided with the IBM Cloud® storage plug-ins.

To order the storage, we must create a PVC. The PVC determines the specification for the storage that we want to provision. After the PVC is created, the storage device and the PV are automatically created for you.

The following image shows how file storage is dynamically provisioned in a cluster. This sample flow works similarly with other storage types, such as block storage.

Sample flow for dynamic provisioning of file storage with the pre-defined silver storage class

The user creates a persistent volume claim (PVC) that specifies the storage type, storage class, size in gigabytes, number of IOPS, and billing type. The storage class determines the type of storage that is provisioned and the allowed ranges for size and IOPS. Creating a PVC in a cluster automatically triggers the storage plug-in for the requested type of storage to provision storage with the given specification.
The storage device is automatically ordered and provisioned into your IBM Cloud infrastructure account. The billing cycle for the storage device starts.
The storage plug-in automatically creates a persistent volume (PV) in the cluster, a virtual storage device that points to the actual storage device in your IBM Cloud infrastructure account.
The PVC and PV are automatically connected to each other. The status of the PVC and the PV changes to Bound. We can now use the PVC to mount persistent storage to the app. If you delete the PVC, the PV and related storage instance are also deleted.

When do I use dynamic provisioning?

Review the following common use cases for dynamic provisioning:

Provision storage when needed: Instead of pre-providing persistent storage for developers and paying for storage that is not used, we can give developers the freedom to provision storage when they need it. To determine the type of storage that the developer can provision, we can define storage classes.
Automate the creation of PVC, PV, and storage device: You want to automatically provision and deprovision storage without manual intervention from a cluster admin.
Create and delete storage often: We have an app or set up a continuous delivery pipeline that creates and removes persistent storage regularly. Persistent storage that is dynamically provisioned with a non-retaining storage class can be removed by deleting the PVC.

For more information about how to dynamically provision persistent storage, see:

Static provisioning

If we have an existing persistent storage device in your IBM Cloud infrastructure account, we can use static provisioning to make the storage instance available to the cluster.

How does it work?

Static provisioning is a feature that is native to Kubernetes and that allows cluster administrators to make existing storage devices available to a cluster. As a cluster administrator, we must know the details of the storage device, its supported configurations, and mount options.

To make existing storage available to a cluster user, we must manually create the storage device, a PV, and a PVC.

The following image shows how to statically provision file storage in a cluster. This sample flow works similar with other storage types, such as block storage.

Sample flow for static provisioning of file storage

The cluster admin gathers all the details about the existing storage device and creates a persistent volume (PV) in the cluster.
Based on the storage details in the PV, the storage plug-in connects the PV with the storage device in your IBM Cloud infrastructure account.
The cluster admin or a developer creates a PVC. Because the PV and the storage device already exist, no storage class is specified in the PVC.
After the PVC is created, the storage plug-in tries to match the PVC to an existing PV. The PVC and the PV match when the same values for the size, IOPS, and access mode are used in the PVC and the PV. When PVC and PV match, the status of the PVC and the PV changes to Bound. We can now use the PVC to mount persistent storage to the app. When you delete the PVC, the PV and the physical storage instance are not removed. You must remove the PVC, PV, and the physical storage instance separately.

When do I use static provisioning?

Review the following common use cases for static provisioning of persistent storage:

Make retained data available to the cluster: You provisioned persistent storage with a retain storage class by using dynamic provisioning. You removed the PVC, but the PV, the physical storage in IBM Cloud infrastructure, and the data still exist. You want to access the retained data from an app in the cluster.
Use an existing storage device: You provisioned persistent storage directly in your IBM Cloud infrastructure account and want to use this storage device in the cluster.
Share persistent storage across clusters in the same zone: You provisioned persistent storage for the cluster. To share the same persistent storage instance with other clusters in the same zone, we must manually create the PV and matching PVC in the other cluster. Note: Sharing persistent storage across clusters is available only if the cluster and the storage instance are located in the same zone.
Share persistent storage across namespaces in the same cluster: You provisioned persistent storage in a namespace of the cluster. You want to use the same storage instance for an app pod that is deployed to a different namespace in the cluster.

For more information about how to statically provision storage, see:

Storage classes

To dynamically provision persistent storage, we must define the type and configuration of the storage that we want.

A Kubernetes storage class is used to abstract the underlying storage platform that is supported in IBM Cloud so that you do not have to know all the details about supported sizes, IOPS, or retention policies to successfully provision persistent storage in a cluster. Red Hat OpenShift on IBM Cloud provides pre-defined storage classes for every type of storage that is supported. Each storage class is designed to abstract the supported storage tier while giving you the choice to decide on the size, IOPS, and retention policy that we want.

For the pre-defined storage class specifications, see:

Not finding what we are looking for? We can also create your own customized storage class to provision the type of storage that we want.

Customizing a storage class

If we cannot use one of the provided storage classes, we can create your own customized storage class. We might want to customize a storage class to specify configurations such as the zone, file system type, server type, or volume binding mode options (block storage only).

Create a customized storage class. We can start by using one of the pre-defined storage classes, or check out our sample customized storage classes.
- Pre-defined storage classes:
- Sample customized storage classes:
  - Classic File Storage
  - Classic Block Storage
Create the customized storage class.
```
oc apply -f <local_file_path>
```
Verify that the customized storage class is created.
```
oc get storageclasses
```
Create a persistent volume claim (PVC) to dynamically provision storage with your customized storage class.
Verify that your PVC is created and bound to a persistent volume (PV). This process might take a few minutes to complete.
```
oc get pvc
```

Changing or updating to a different storage class

When you dynamically provision persistent storage by using a storage class, you provision persistent storage with a specific configuration. We cannot change the name of the storage class or the type of storage that you provisioned. However, we have the option to scale your storage as shows in the following table.

Storage solution	Scaling options
Classic File Storage	We can increase your storage size and assigned IOPS by modifying the existing volume.
Classic Block Storage	We can increase your storage size and assigned IOPS by modifying the existing volume.
IBM Cloud Object Storage	Your volume automatically scales in size and we are charged based on your actual consumption. However, we cannot change the performance attributes of our volume as they are defined in the storage class that you used to create your bucket in IBM Cloud Object Storage. To change to a different storage class, we must provision a new bucket by using the storage class that we want. Then, copy your data from the old bucket to the new one.
Portworx	We can increase your storage size by changing your PVC specifications .

Changing the default storage class

We can change the default storage class that a persistent volume (PV) uses if no storage class is specified in the persistent volume claim (PVC). We can have only one default storage class. The default storage classes vary by the Kubernetes version of the cluster. When you update a cluster from an earlier version of Kubernetes, the default storage class does not change.

Kubernetes 1.17 and later: ibmc-file-gold
Kubernetes 1.16 and earlier: ibmc-file-bronze

To change the default storage class for a cluster:

Log in to your account. If applicable, target the appropriate resource group. Set the context for the cluster.
List available storage classes. Note the name of the storage class that we want to make the default, and the current default storage class that has (default) in the Name.
```
oc get storageclasses
```

Create a default storage class, replacing <storageclass> with the storage class that we want to use.

oc patch storageclass <storageclass> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Remove the previous default storage class.

oc patch storageclass <previous_default_storageclass> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

Verify that the default storage class is set.
```
oc get storageclasses | grep "(default)"
```