ceph-csi-rbd

The ceph-csi-rbd chart adds rbd volume support to your cluster.

Install from release repo

Add chart repository to install helm charts from it

helm repo add ceph-csi https://ceph.github.io/csi-charts

Install from local Chart

we need to enter into the directory where all charts are present

cd charts

Note: charts directory is present in root of the ceph-csi project

Install chart

To install the Chart into your Kubernetes cluster

  • For helm 2.x

    helm install --namespace "ceph-csi-rbd" --name "ceph-csi-rbd" ceph-csi/ceph-csi-rbd
    
  • For helm 3.x

    Create the namespace where Helm should install the components with

    kubectl create namespace "ceph-csi-rbd"
    

    Run the installation

    helm install --namespace "ceph-csi-rbd" "ceph-csi-rbd" ceph-csi/ceph-csi-rbd
    

After installation succeeds, you can get a status of Chart

helm status --namespace "ceph-csi-rbd" "ceph-csi-rbd"

Upgrade Chart

If you want to upgrade your Chart, use the following commands.

helm repo update ceph-csi
helm upgrade --namespace ceph-csi-rbd ceph-csi-rbd ceph-csi/ceph-csi-rbd

For upgrading to a specific version, provide the flag --version and the version.

Do not forget to include your values, if they differ from the default values. We recommend not to use --reuse-values in case there are new defaults AND compare your currently used values with the new default values.

Known Issues Upgrading

  • When upgrading to version >=3.7.0, you might encounter an error that the RBD CSI Driver cannot be updated. Please refer to issue for more details. This is due to the CSIDriver resource not being updatable. To work around this you can delete the CSIDriver object by running:

    kubectl delete csidriver rbd.csi.ceph.com
    

    Then rerun your helm upgrade command.

Delete Chart

If you want to delete your Chart, use this command

  • For helm 2.x

    helm delete --purge "ceph-csi-rbd"
    
  • For helm 3.x

    helm uninstall "ceph-csi-rbd" --namespace "ceph-csi-rbd"
    

If you want to delete the namespace, use this command

kubectl delete namespace ceph-csi-rbd

Configuration

The following table lists the configurable parameters of the ceph-csi-cephfs charts and their default values.

ParameterDescriptionDefault
rbac.createSpecifies whether RBAC resources should be createdtrue
serviceAccounts.nodeplugin.createSpecifies whether a nodeplugin ServiceAccount should be createdtrue
serviceAccounts.nodeplugin.nameThe name of the nodeplugin ServiceAccount to use. If not set and create is true, a name is generated using the fullname""
serviceAccounts.provisioner.createSpecifies whether a provisioner ServiceAccount should be createdtrue
serviceAccounts.provisioner.nameThe name of the provisioner ServiceAccount to use. If not set and create is true, a name is generated using the fullname""
csiConfigConfiguration for the CSI to connect to the cluster[]
csiMappingConfiguration details of clusterID,PoolID,FscID mapping[]
encryptionKMSConfigConfiguration for the encryption KMS{}
commonLabelsLabels to apply to all resources{}
logLevelSet logging level for csi containers. Supported values from 0 to 5. 0 for general useful logs, 5 for trace level verbosity.5
sidecarLogLevelSet logging level for csi sidecar containers. Supported values from 0 to 5. 0 for general useful logs, 5 for trace level verbosity.1
nodeplugin.nameSpecifies the nodeplugins namenodeplugin
nodeplugin.updateStrategySpecifies the update Strategy. If you are using ceph-fuse client set this value to OnDeleteRollingUpdate
nodeplugin.priorityClassNameSet user created priorityclassName for csi plugin pods. default is system-node-critical which is highest prioritysystem-node-critical
nodeplugin.imagePullSecretsSpecifies imagePullSecrets for containers[]
nodeplugin.profiling.enabledSpecifies whether profiling should be enabledfalse
nodeplugin.registrar.image.repositoryNode Registrar image repository URLregistry.k8s.io/sig-storage/csi-node-driver-registrar
nodeplugin.registrar.image.tagImage tagv2.10.0
nodeplugin.registrar.image.pullPolicyImage pull policyIfNotPresent
nodeplugin.plugin.image.repositoryNodeplugin image repository URLquay.io/cephcsi/cephcsi
nodeplugin.plugin.image.tagImage tagcanary
nodeplugin.plugin.image.pullPolicyImage pull policyIfNotPresent
nodeplugin.nodeSelectorKubernetes nodeSelector to add to the Daemonset{}
nodeplugin.tolerationsList of Kubernetes tolerations to add to the Daemonset{}
nodeplugin.podSecurityPolicy.enabledIf true, create & use Pod Security Policy resources.false
provisioner.nameSpecifies the name of provisionerprovisioner
provisioner.replicaCountSpecifies the replicaCount3
provisioner.defaultFSTypeSpecifies the default Fstypeext4
provisioner.deployControllerIt enables or disables the deployment of controller which generates the OMAP data if it is not presenttrue
provisioner.hardMaxCloneDepthHard limit for maximum number of nested volume clones that are taken before a flatten occurs8
provisioner.softMaxCloneDepthSoft limit for maximum number of nested volume clones that are taken before a flatten occurs4
provisioner.maxSnapshotsOnImageMaximum number of snapshots allowed on rbd image without flattening450
provisioner.minSnapshotsOnImageMinimum number of snapshots allowed on rbd image to trigger flattening250
provisioner.skipForceFlattenSkip image flattening if kernel support mapping of rbd images which has the deep-flatten featurefalse
provisioner.timeoutGRPC timeout for waiting for creation or deletion of a volume60s
provisioner.clusternameCluster name to set on the RBD image""
provisioner.setmetadataSet metadata on volumetrue
provisioner.priorityClassNameSet user created priorityclassName for csi provisioner pods. Default is system-cluster-critical which is less priority than system-node-criticalsystem-cluster-critical
provisioner.enableHostNetworkSpecifies whether hostNetwork is enabled for provisioner pod.false
provisioner.imagePullSecretsSpecifies imagePullSecrets for containers[]
provisioner.profiling.enabledSpecifies whether profiling should be enabledfalse
provisioner.provisioner.image.repositorySpecifies the csi-provisioner image repository URLregistry.k8s.io/sig-storage/csi-provisioner
provisioner.provisioner.image.tagSpecifies image tagv4.0.0
provisioner.provisioner.image.pullPolicySpecifies pull policyIfNotPresent
provisioner.provisioner.image.extraArgsSpecifies extra arguments for the provisioner sidecar[]
provisioner.snapshotter.args.enableVolumeGroupSnapshotsenables the creation of volume group snapshotsfalse
provisioner.attacher.image.repositorySpecifies the csi-attacher image repository URLregistry.k8s.io/sig-storage/csi-attacher
provisioner.attacher.image.tagSpecifies image tagv4.5.
provisioner.attacher.image.pullPolicySpecifies pull policyIfNotPresent
provisioner.attacher.image.extraArgsSpecifies extra arguments for the attacher sidecar[]
provisioner.attacher.nameSpecifies the name of csi-attacher sidecarattacher
provisioner.attacher.enabledSpecifies whether attacher sidecar is enabledtrue
provisioner.resizer.image.repositorySpecifies the csi-resizer image repository URLregistry.k8s.io/sig-storage/csi-resizer
provisioner.resizer.image.tagSpecifies image tagv1.10.0
provisioner.resizer.image.pullPolicySpecifies pull policyIfNotPresent
provisioner.resizer.image.extraArgsSpecifies extra arguments for the resizer sidecar[]
provisioner.resizer.nameSpecifies the name of csi-resizer sidecarresizer
provisioner.resizer.enabledSpecifies whether resizer sidecar is enabledtrue
provisioner.snapshotter.image.repositorySpecifies the csi-snapshotter image repository URLregistry.k8s.io/sig-storage/csi-snapshotter
provisioner.snapshotter.image.tagSpecifies image tagv7.0.0
provisioner.snapshotter.image.pullPolicySpecifies pull policyIfNotPresent
provisioner.snapshotter.image.extraArgsSpecifies extra arguments for the snapshotter sidecar[]
provisioner.nodeSelectorSpecifies the node selector for provisioner deployment{}
provisioner.tolerationsSpecifies the tolerations for provisioner deployment{}
provisioner.affinitySpecifies the affinity for provisioner deployment{}
provisioner.podSecurityPolicy.enabledSpecifies whether podSecurityPolicy is enabledfalse
topology.enabledSpecifies whether topology based provisioning support should be exposed by CSIfalse
topology.domainLabelsDomainLabels define which node labels to use as domains for CSI nodeplugins to advertise their domains{}
readAffinity.enabledEnable read affinity for RBD volumes. Recommended to set to true if running kernel 5.8 or newer.false
readAffinity.crushLocationLabelsDefine which node labels to use as CRUSH location. This should correspond to the values set in the CRUSH map. For more information, click here[]
provisionerSocketFileThe filename of the provisioner socketcsi-provisioner.sock
pluginSocketFileThe filename of the plugin socketcsi.sock
kubeletDirkubelet working directory/var/lib/kubelet
cephLogDirHostPathHost path location for ceph client processes logging, ex: rbd-nbd/var/log/ceph
driverNameName of the csi-driverrbd.csi.ceph.com
configMapNameName of the configmap which contains cluster configurationceph-csi-config
externallyManagedConfigmapSpecifies the use of an externally provided configmapfalse
cephConfConfigMapNameName of the configmap which contains ceph.conf configurationceph-config
kmsConfigMapNameName of the configmap used for encryption kms configurationceph-csi-encryption-kms-config
storageClass.createSpecifies whether the StorageClass should be createdfalse
storageClass.nameSpecifies the rbd StorageClass namecsi-rbd-sc
storageClass.annotationsSpecifies the annotations for the rbd StorageClass[]
storageClass.clusterIDString representing a Ceph cluster to provision storage from<cluster-ID>
storageClass.dataPoolSpecifies the erasure coded pool""
storageClass.poolCeph pool into which the RBD image shall be createdreplicapool
storageclass.imageFeaturesSpecifies RBD image featureslayering
storageclass.tryOtherMountersSpecifies whether to try other mounters in case if the current mounter fails to mount the rbd image for any reasonfalse
storageclass.mkfsOptionsOptions to pass to the mkfs command while creating the filesystem on the RBD device""
storageClass.mounterSpecifies RBD mounter""
storageClass.cephLogDirceph client log location, it is the target bindmount path used inside container"/var/log/ceph"
storageClass.cephLogStrategyceph client log strategy, available options remove or compress or preserve"remove"
storageClass.volumeNamePrefixPrefix to use for naming RBD images""
storageClass.encryptedSpecifies whether volume should be encrypted. Set it to true if you want to enable encryption""
storageClass.encryptionKMSIDSpecifies the encryption kms id""
storageClass.topologyConstrainedPoolsAdd topology constrained pools configuration, if topology based pools are setup, and topology constrained provisioning is required[]
storageClass.mapOptionsSpecifies comma-separated list of map options""
storageClass.unmapOtpionsSpecifies comma-separated list of unmap options""
storageClass.stripeUnitSpecifies the stripe unit in bytes""
storageClass.stripeCountSpecifies the number of objects to stripe over before looping""
storageClass.objectSizeSpecifies the object size in bytes""
storageClass.provisionerSecretThe secrets have to contain user and/or Ceph admin credentials.csi-rbd-secret
storageClass.provisionerSecretNamespaceSpecifies the provisioner secret namespace""
storageClass.controllerExpandSecretSpecifies the controller expand secret namecsi-rbd-secret
storageClass.controllerExpandSecretNamespaceSpecifies the controller expand secret namespace""
storageClass.nodeStageSecretSpecifies the node stage secret namecsi-rbd-secret
storageClass.nodeStageSecretNamespaceSpecifies the node stage secret namespace""
storageClass.fstypeSpecify the filesystem type of the volumeext4
storageClass.reclaimPolicySpecifies the reclaim policy of the StorageClassDelete
storageClass.allowVolumeExpansionSpecifies whether volume expansion should be allowedtrue
storageClass.mountOptionsSpecifies the mount options for storageClass[]
secret.createSpecifies whether the secret should be createdfalse
secret.nameSpecifies the rbd secret namecsi-rbd-secret
secret.userIDSpecifies the user ID of the rbd secret<plaintext ID>
secret.userKeySpecifies the key that corresponds to the userID<Ceph auth key corresponding to ID above>
secret.encryptionPassphraseSpecifies the encryption passphrase of the secrettest_passphrase
selinuxMountMount the host /etc/selinux inside pods to support selinux-enabled filesystemstrue
CSIDriver.fsGroupPolicySpecifies the fsGroupPolicy for the CSI driver objectFile
CSIDriver.seLinuxMountSpecify for efficient SELinux volume relabelingtrue

Command Line

You can pass the settings with helm command line parameters. Specify each parameter using the --set key=value argument to helm install. For Example:

helm install --set configMapName=ceph-csi-config --set provisioner.podSecurityPolicy.enabled=true