Skip to content

[terraform/aws] Add optional EKS node monitoring agent add-on - #1784

Merged
barroco merged 2 commits into
interuss:masterfrom
StevenPG:stevenpg-aws-node-monitoring-agent
Oct 19, 2026
Merged

barroco merged 2 commits into
interuss:masterfrom
StevenPG:stevenpg-aws-node-monitoring-agent

Conversation

@StevenPG

Copy link
Copy Markdown
Contributor

Part of #1782.

Changes

Add the EKS node monitoring agent add-on (eks-node-monitoring-agent), controlled by a new aws_enable_node_monitoring_agent variable that defaults to false. The agent reports additional node health conditions (container runtime, kernel, networking, storage) which EKS node auto repair, when enabled on the node group (see #1778), uses to detect unhealthy nodes that still report Ready.

It is off by default because it runs as a privileged DaemonSet with the NET_ADMIN capability, which some security requirements do not allow. The add-on requires no IAM role and no configuration. Enabling it on an existing cluster only adds the DaemonSet: nodes and workloads are not restarted. Operators who installed the agent manually must uninstall it first (noted in NEXT_RELEASE_NOTES.md).

Testing

  • terraform fmt -recursive -check deploy
  • terraform validate on terraform-aws-dss
  • generate_terraform_variables.sh --lint

Applied to a live cluster

Enabled on an existing EKS 1.34 deployment (3 nodes, CockroachDB, us-west-1). The plan contained a single add-on creation, which applied in about 25s. The agent runs on every node and reports KernelReady, StorageReady, ContainerRuntimeReady and NetworkingReady (all True), with no restart of nodes or workloads.

EKS node auto repair only acts on the standard node conditions unless
the EKS node monitoring agent is installed. The agent reports
additional conditions (container runtime, kernel, networking, storage)
so that auto repair, when enabled on the node group, can detect
unhealthy nodes which still report `Ready`.

This change installs the agent through the `eks-node-monitoring-agent`
EKS add-on, behind a new variable `aws_enable_node_monitoring_agent`
exposed by the `terraform-aws-dss` module.

The variable defaults to `false`: the agent runs as a privileged
DaemonSet with the `NET_ADMIN` capability, which some security
requirements do not allow. Operators can opt in after review.

The add-on requires no IAM role and no configuration. Enabling it on an
existing cluster only creates the DaemonSet once the node group is
available: nodes and workloads are not restarted. Clusters where the
agent was installed manually must uninstall it first, otherwise the
creation of the add-on fails because of conflicting resources. This is
mentioned in the release notes.

Part of interuss#1782.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

@barroco barroco left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @StevenPG!

@barroco
barroco merged commit af44bf8 into interuss:master Oct 19, 2026
20 checks passed
StevenPG added a commit to StevenPG/dss that referenced this pull request Oct 21, 2026
… agent add-on

Resolve merge conflicts with interuss#1784, which added the adjacent
aws_enable_node_monitoring_agent variable. Both variables are kept in
variables.py, terraform-aws-dss/main.tf and NEXT_RELEASE_NOTES.md, and
the *.gen.tf / TFVARS.gen.md files were regenerated.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants