Skip to content

fix: allow enabling/disabling host subscriptions for driver pod - #3106

Closed
rahulait wants to merge 1 commit into
NVIDIA:mainfrom
rahulait:fix-host-subscription
Closed

rahulait wants to merge 1 commit into
NVIDIA:mainfrom
rahulait:fix-host-subscription

Conversation

@rahulait

@rahulait rahulait commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Alternative implementation. Fixes #2817 by making host subscription mounts independent of custom repository configuration.

Previously, configuring driver.repoConfig implicitly disabled RHEL host subscription mounts. This prevented driver installations that require both host subscription repositories and additional custom repositories.

Changes

Add driver.hostSubscription.enabled to Helm, ClusterPolicy, and NVIDIADriver:

Configuration Behavior
Omitted Preserve automatic mounts for eligible OS/runtime combinations
true Mount standard subscription paths, retaining eligibility checks
false Skip host subscription mounts

Use a pointer boolean to distinguish omitted values from explicit false. Custom repositories no longer implicitly disable subscriptions.

The hostSubscription object also leaves room for future configuration, such as custom subscription paths. This change only introduces enabled; custom paths are not currently supported.

Example

Air-gapped installations can explicitly disable host subscription mounts:

driver:
  repoConfig:
    configMapName: custom-repos
  hostSubscription:
    enabled: false

Checklist

  • No secrets, sensitive information, or unrelated changes
  • Lint checks passing (make lint)
  • Generated assets in-sync (make validate-generated-assets)
  • Go mod artifacts in-sync (make validate-modules)
  • Test cases are added for new code paths

Testing


Devin Review

@rahulait
rahulait requested a review from a team as a code owner October 21, 2026 20:04
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

📝 Walkthrough

Both API versions and Helm values add host subscription mount configuration, enabled by default and disabled only when explicitly set to false. Runtime logic uses this setting to control subscription mounts while keeping custom repository mounts independent. Tests cover API serialization, deep-copy behavior, and mount combinations across operating systems and runtimes.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: 🟡 Moderate · up to 4d370

Existing RHEL installs that use a custom repository mirror, especially air-gapped ones, may find driver pods failing to start after upgrade unless they set hostSubscription.enabled: false. That opt-out also does nothing when the NVIDIADriver custom resource mode is used. Fix the Helm template and document or reconsider the default before merging.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments: 1
deployments/gpu-operator/values.yaml: 1

244-245: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Render hostSubscription in the NVIDIADriver template.

When driver.nvidiaDriverCRD.enabled is true, templates/clusterpolicy.yaml skips the driver block that renders driver.hostSubscription. The default templates/nvidiadriver.yaml renders repoConfig but does not render driver.hostSubscription, so driver.hostSubscription.enabled has no effect in that supported install mode.

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/gpu-operator/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: 001468f8-3af8-4c40-962b-f41e647c92b7
📥 Commits

Reviewing files that changed from the base of the PR and between 932f906 and 4d37056.

⛔ Files ignored due to path filters: 8
  • api/nvidia/v1/zz_generated.deepcopy.go is excluded by !**/zz_generated.*.go
  • api/nvidia/v1alpha1/zz_generated.deepcopy.go is excluded by !**/zz_generated.*.go
  • bundle/manifests/nvidia.com_clusterpolicies.yaml is excluded by !bundle/manifests/nvidia.com_*.yaml
  • bundle/manifests/nvidia.com_nvidiadrivers.yaml is excluded by !bundle/manifests/nvidia.com_*.yaml
  • config/crd/bases/nvidia.com_clusterpolicies.yaml is excluded by !config/crd/bases/**
  • config/crd/bases/nvidia.com_nvidiadrivers.yaml is excluded by !config/crd/bases/**
  • deployments/gpu-operator/crds/nvidia.com_clusterpolicies.yaml is excluded by !deployments/gpu-operator/crds/**
  • deployments/gpu-operator/crds/nvidia.com_nvidiadrivers.yaml is excluded by !deployments/gpu-operator/crds/**
📒 Files selected for processing: 10
  • api/nvidia/v1/clusterpolicy_types.go
  • api/nvidia/v1/clusterpolicy_types_test.go
  • api/nvidia/v1alpha1/nvidiadriver_types.go
  • api/nvidia/v1alpha1/nvidiadriver_types_test.go
  • controllers/object_controls.go
  • controllers/transforms_test.go
  • deployments/gpu-operator/templates/clusterpolicy.yaml
  • deployments/gpu-operator/values.yaml
  • internal/state/driver_test.go
  • internal/state/driver_volumes.go

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Comment on lines +242 to +245
# Mount host subscription files on eligible OS/runtime combinations, including with repoConfig.
# Set to false for air-gapped hosts without subscription files.
hostSubscription:
enabled: true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

The default changes behavior on helm upgrade for RHEL installs that use repoConfig.

The change summary says that, before this PR, RHEL with repoConfig skipped subscription mounts. With enabled: true as the default, the same installs now mount /etc/pki/entitlement, /etc/yum.repos.d/redhat.repo and /etc/rhsm. This applies to vanilla Kubernetes with a non-CRI-O runtime.

The SubscriptionPathMap entries use HostPathDirectory and HostPathFile. A host without these paths fails the mount, and the driver pod stays in ContainerCreating. The paths are missing on air-gapped hosts that use a mirror in repoConfig.

Because the driver daemonset rolls with maxUnavailable, the failure can stall the driver rollout after the operator upgrade.

The workaround is hostSubscription.enabled: false. Existing users do not know to set it. Either document this as a breaking default in the upgrade or release notes, or keep the old behavior for existing installs. For example, leave the key unset by default so the operator decides.

As per path instructions: "any default whose change alters behaviour for existing installs on helm upgrade."

Source: Path instructions

Signed-off-by: Rahul Sharma <rahulsharm@nvidia.com>
@rahulait
rahulait force-pushed the fix-host-subscription branch from 4d37056 to a14a903 Compare October 21, 2026 20:09

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Devin Review

Comment on lines +3781 to +3782
if !config.Driver.HostSubscription.IsEnabled() {
n.logger.Info("Skipping host subscription mounts because hostSubscription is disabled", "OS", osID)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Existing air-gapped RHEL driver pods fail

On RHEL hosts with repoConfig but missing subscription files, omitted hostSubscription now adds mandatory hostPath mounts. Existing driver pods fail to start after upgrade.

Learn more

Both driver render paths previously omitted RHEL subscription mounts when a custom repository was configured. The new default enables those mounts even for existing custom-repository installations. The subscription paths require host files and directories, so Kubernetes cannot start the pod on an air-gapped node where they do not exist. The same condition changed in getDriverAdditionalConfigs.

Example: A RHEL node has a custom repository ConfigMap but no /etc/pki/entitlement. Before upgrade its driver pod starts without subscription mounts. After upgrade, its unchanged resource produces a required /etc/pki/entitlement hostPath and the driver pod cannot start.

Recommended fix: Preserve the old skip behavior when hostSubscription.enabled is omitted and repoConfig is configured on RHEL, in both transformDriverContainer and getDriverAdditionalConfigs. Allow explicit true to mount both and explicit false to skip. If changing the omitted default is necessary, provide a migration that keeps existing installations running.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +205 to +207
{{- if .Values.driver.hostSubscription }}
hostSubscription: {{ toYaml .Values.driver.hostSubscription | nindent 6 }}
{{- end }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Helm switch leaves default driver mounts enabled

With nvidiaDriverCRD.enabled=true, driver.hostSubscription.enabled=false reaches ClusterPolicy but not the default NVIDIADriver. That driver still mounts subscription paths, so air-gapped RHEL pods fail to start.

Learn more

The Helm chart has two driver rendering modes. When driver.nvidiaDriverCRD.enabled is true, the driver specification comes from the default NVIDIADriver template, while ClusterPolicy does not supply its driver settings. The newly added value is emitted only in the ClusterPolicy template. The default NVIDIADriver omits it, so its IsEnabled() call selects mounts even when the Helm value is false.

Example: Install with driver.nvidiaDriverCRD.enabled=true, driver.nvidiaDriverCRD.deployDefaultCR=true, driver.hostSubscription.enabled=false, and a RHEL host without /etc/pki/entitlement. The default driver resource omits the setting and the rendered pod requires that path; the driver pod cannot start.

Recommended fix: Also emit hostSubscription under spec in templates/nvidiadriver.yaml, using the same values key. Test both Helm driver modes with an explicit false value.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

@rahulait

Copy link
Copy Markdown
Contributor Author

Closing in favor of #3109

@rahulait rahulait closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Cannot use custom repoConfig together with RHEL host subscription repositories after #2391

1 participant