Volume condition reporting on EBS CSI volumes enhancement

[abeowlu]

Is your feature request related to a problem?/Why is this needed
Implement enhancement for volume monitoring in the EBS CSI driver. Where volume condition can be bubbled up to kubelet to report abnormal or healthy condition volumes to cluster admins at a glance using prometheus metrics

/feature

Describe the solution you'd like in detail
Similar to the enhancement on the EFS CSI Driver;

• enable volume condition capabilities in the node server
• check that the EBS volume is responsive to IO and report health back to kubelet - preferably with a configurable reporting heartbeat, a default of 120sec to start
• for stuck volume an auto-recovery attempt before giving up and marking volume condition as abnormal would be the highlight of this enhancement
• configure kubelet to request CSI volume condition - and push this as prometheus metrics for cluster admins

Describe alternatives you've considered

• creating alternate workflow that checks the volume mounts on a node and fails at any attempt to attempt self-healing on dead volume mounts

Additional context

• similar enhancement on EFS CSI Driver

[ConnorJC3]

Hi, thanks for the feature request! I don't have a firm ETA available at this time but I've made sure this request is on our team's backlog.

[ConnorJC3]

/kind feature

[abeowlu]

Thanks @ConnorJC3 , I can contribute this feature to the project if that is alright

[ConnorJC3]

SIG Storage is currently re-working the volume health monitoring feature in upstream Kubernetes, so please hold off for now. One of our team members is actively engaged in those discussions and the design of the Kubernetes feature is likely to change significantly.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[abeowlu]

Hi @ConnorJC3 , I followed up on the KEP release on the SIG storage channel today
The API spec proposed for this is already in alpha, do you think it is reasonable to release this in the Addon as opt in alpha?

[ConnorJC3]

/lifecycle frozen

We're in discussion about this, and one of our maintainers @torredil is working with upstream on the alpha KEP. I don't have specific details right now because the priority is getting the KEP out upstream first but we will hopefully have something to share relatively soon.