From 762181ae4918e3e261df06ed983a1ef1cb61b10c Mon Sep 17 00:00:00 2001 From: Michael Still Date: Fri, 14 Aug 2026 06:22:22 +1000 Subject: Extend the nova_scheduler healthcheck timeout. Nova changed in 2026.1 how long their services take to gracefully shutdown. The default value of [DEFAULT]/graceful_shutdown_timeout has been changed from 60 to 180 seconds for all Nova services. [...] Operators using external system (e.g. k8s, systemd) to manage the Nova services should ensure that their service stop timeouts are set to at least graceful_shutdown_timeout to avoid forcefully killing service before Nova finish its graceful shutdown. Kolla-Ansible failed to update the period of time that ansible waits for Nova services to shut down. This causes both CI failures and failures for deployers when trying to manage some Nova containers. Prompt: Michael asked for assistance debugging why CI jobs were failing with nova-scheduler timeouts. Assisted-By: Claude Opus 5 (1M context, high effort) Co-Authored-By: Claude Opus 5 (1M context, high effort) Closes-Bug: #2162860 Change-Id: If7c5a6d885b19e39cfc305440424f10de3a5ea05 Signed-off-by: Michael Still (cherry picked from commit 4ce86ef1db631c15e6e5e142d2908f73348377d4) --- ansible/roles/nova/defaults/main.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'ansible') diff --git a/ansible/roles/nova/defaults/main.yml b/ansible/roles/nova/defaults/main.yml index 215826cdd..020048781 100644 --- a/ansible/roles/nova/defaults/main.yml +++ b/ansible/roles/nova/defaults/main.yml @@ -168,7 +168,11 @@ nova_metadata_healthcheck: nova_scheduler_enable_healthchecks: "{{ enable_container_healthchecks }}" nova_scheduler_healthcheck_interval: "{{ default_container_healthcheck_interval }}" -nova_scheduler_healthcheck_retries: "{{ default_container_healthcheck_retries }}" + +# NOTE(mikal): this one is deliberately overridden because of the issues discussed +# in releasenotes/notes/nova-scheduler-healthcheck-retries-12e681b16f6322aa.yaml +nova_scheduler_healthcheck_retries: 8 + nova_scheduler_healthcheck_start_period: "{{ default_container_healthcheck_start_period }}" nova_scheduler_healthcheck_test: ["CMD-SHELL", "healthcheck_port nova-scheduler {{ om_rpc_port }}"] nova_scheduler_healthcheck_timeout: "{{ default_container_healthcheck_timeout }}" -- cgit v1.2.3