aboutsummaryrefslogtreecommitdiff
path: root/ansible
AgeCommit message (Collapse)AuthorFilesLines
2026-07-22[stable-only] Split CA of ProxySQL internal TLS and backend TLSSeunghun Lee4-1/+35
ProxySQL frontend TLS (database internal TLS) and backend TLS (database backend TLS) expects different CA if users use chain of certificates (e.g. use of intermediate certificates). Frontend TLS expects the intermediate certificate that signed the leaf certificate as a CA file, whereas backend TLS expects root CA. To fix, users are now required to create a file named ``proxysql-ca.pem`` at ``kolla_certificates_dir``, which is a copy of the intermediate certificate. If your system is not affected by this bug, you're not required to create the new file ``proxysql-ca.pem``. Closes-bug: #2150449 Change-Id: I88911badb0d0801f79317df6681c571cc1c8cfee Signed-off-by: Seunghun Lee <seunghun@stackhpc.com>
2026-07-18Add openstack cacert to uwsgi env for GlanceAlex Welsh1-0/+4
This change ensures REQUESTS_CA_BUNDLE and SSL_CERT_FILE environment variables are exported in uWSGI conf, fixing SSL validation errors in Glance. By default it doesn't inherit from the host, so these must be included explicitly, otherwise internal communication between glance api instances will always fail when internal TLS is enabled. Change-Id: Ifbe3547ba6b933bb1e4d5599be94ba428c4f49c5 Closes-Bug: #2160218 Signed-off-by: Alex Welsh <alex@stackhpc.com> (cherry picked from commit 9c41bdca79d4c7267beeece141909e679f70dea8)
2026-07-14Fix nova-compute startup after reprovisioningPierre Riteau2-0/+66
We need to restore the /var/lib/nova/compute_id file to avoid nova-compute failing to start with the following error: nova.exception.InvalidConfiguration: No local node identity found, but this is not our first startup on this host. Refusing to start after potentially having lost that state! K-A now query IDs of hypervisors by their hostname and create necessary compute_id file under node config directory for nova-compute before starting nova-compute containers. The compute_id file gets copied under /var/lib/nova/ on container start up. Co-Authored-By: bjorn lofdahl <bjorn.lofdahl@gmail.com> Co-Authored-By: Seunghun Lee <seunghun@stackhpc.com> Co-Authored-By: Owen Jones <owen@stackhpc.com> Change-Id: If635577fa240f0953ad02d2c28473480d9ad011d Closes-Bug: #2051011 Signed-off-by: Owen Jones <owen@stackhpc.com> Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com> (cherry picked from commit 3e4cfd49da38c7db2ac3ad47a75dc89b14cbb513)
2026-07-09Merge "Fix typo in neutron group vars" into stable/2026.1Zuul1-1/+1
2026-07-09prometheus: Fix Grafana blackbox endpoint checksDawud1-2/+4
Split the Grafana blackbox endpoint into separate public and internal checks. The public endpoint is now only monitored when both Grafana and ``enable_grafana_external`` are enabled (which is the default when enabling Grafana), matching the HAProxy external listener check. The internal endpoint remains monitored when Grafana is enabled and either external Grafana is disabled or the internal and external VIPs are separate. This avoids losing Grafana monitoring when ``enable_grafana_external`` is disabled. Closes-Bug: #2157003 Change-Id: Iea3b21dcfa4284f2a74c1bee95577ce8a219cf5f Signed-off-by: Dawud <dawud@stackhpc.com> (cherry picked from commit 6e9bea3291a55829b32d58613b0414af547ec4a1)
2026-07-09Fix typo in neutron group varsNicolas MAILLET-CONTOZ1-1/+1
Closes-Bug: 2160035 Change-Id: I2f3ef4ef1085c93fd7e697c03fd530885795bcc4 Signed-off-by: alma.maillet-contoz <alma.maillet-contoz@infomaniak.com> (cherry picked from commit 4784eee7086043610c49c3f06d5e77eb816c9a2e)
2026-07-09Fix j2lint jinja-statements-indentation errorPierre Riteau2-4/+2
Change-Id: I109f80d8903cb6f9c608ab39d181a001d426fd87 Signed-off-by: Pierre Riteau <pierre@stackhpc.com> (cherry picked from commit b2e63e2791e49cdca206241e07f27a53f4ac7556)
2026-07-07Increase Glance `chunked-input-limit`Alex Welsh2-0/+4
This change sets the default Glance uWSGI `chunked-input-limit` to 67108864 (64MB). Before this change, Glance would use the uWSGI default 1MB. Snapshotting an instance would hang indefinitely because Nova sends larger chunks. Change-Id: I43f9232efade5e98bcd67b9c94e7161c790995ce Closes-Bug: #2158974 Signed-off-by: Alex Welsh <alex@stackhpc.com> (cherry picked from commit a64dd7075780d6b614aabff7769f4f83a315b680)
2026-07-01UnicodeDecodeError in PodmanWorker.compare_configEttore Simone1-2/+6
During container config comparison, podman's Docker-compat exec API inconsistently demultiplexes the 8-byte frame headers from the output stream depending on podman version, leaving binary bytes in raw_output. This will raise an "UnicodeDecodeError: 'utf-8' codec can't decode byte 0x?? in position 167: invalid start byte" Closes-Bug: 2153943 Change-Id: Ida1c017f50b9a24b88f8962d304eb015fa357211 Signed-off-by: Ettore Simone <ettore.simone@gmail.com> (cherry picked from commit c99f521356012d5bc42ab3c41a994b67657e86f0)
2026-06-29Tune libvirtd connection limits for OpenStackMichael Still2-0/+21
The libvirt daemon ships with max_workers=20 and max_client_requests=5. Those defaults suit a multi-tenant host serving many mutually distrustful clients, where the per-client request cap protects the daemon from any single client monopolising it. Under Kolla-Ansible, nova-compute is effectively the sole privileged client of libvirtd, so the cap of 5 concurrent requests per client becomes the binding constraint rather than a safety net. When several instances build at once -- as they do during the tempest run of the upgrade jobs -- nova-compute's concurrent libvirt RPCs (instance defineXML calls plus the resource-tracker periodic) exceed five. Further requests queue, the connection's keepalive responses are starved, and after keepalive_interval (5s) x keepalive_count (5) = 25s libvirtd closes the connection. nova then sees "Cannot recv data: Input/output error" mid-defineXML, the build is rescheduled, and once all hosts are exhausted the instance is left in ERROR. This presents as flaky upgrade-job failures. Example failure, on the voting kolla-ansible-debian-bookworm-upgrade job (stable/2025.2): https://zuul.opendev.org/t/openstack/build/73ff5a97567d4e05b1774170e0f4aada with primary/logs/kolla/libvirt/libvirtd.txt showing libvirt's own recommendation to tune this parameter: warning : Client hit max requests limit 5. This may result in keep-alive timeouts. Consider tuning the max_client_requests server parameter ... error : connection closed due to keepalive timeout Raise the per-client cap to 20 -- comfortably above the handful of concurrent builds plus periodics seen in practice -- and the worker pool to 50 so that lifting the per-client limit does not starve other connections such as live migration, health checks and periodic tasks. Both values are exposed as variables (libvirt_max_client_requests and libvirt_max_workers) so operators can tune them further. This is not a Kolla-specific workaround -- the stock libvirtd.conf comment warns that setting max_client_requests too low "may cause keepalive timeouts", and since 2022 libvirtd emits the runtime warning quoted above. As Daniel Berrange explained when he added that warning, the keepalive packets "are subject to the 'max_client_requests' limit just like any API calls", so a client that saturates the limit can have its connection dropped: https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/5NIJPA7SJVY55VSK7RTEDEHKWOZYJKT6/ Red Hat documents the same tuning for libvirt hosts under load in its knowledge base, notably "Too many active clients are dropping the connection" at https://access.redhat.com/solutions/28146. Assisted-By: Claude Opus 4.8 (1M context, medium effort) Partial-Bug: 2158515 Change-Id: I131c3d4d57dc73ca4dd421cc8f28465248c9830e (cherry picked from commit 5acd2f91d22213ef04bd6988a2a0261982b0cf05) Signed-off-by: Michael Still <mikal@stillhq.com>
2026-06-25Start virtlogd in nova-libvirt and route console logs to it.Michael Still3-2/+46
Due to implementation choices in qemu and libvirt, standard log rotate does not work on these console files, and instead the usual approach is to have libvirt's virtlogd rotate the contents of the console log file when it gets a bit big. OpenStack Compute knows how to read data from more than one rotated console log file if required to service a REST API request. OpenStack Kolla and Kolla-Ansible deploy OpenStack components in docker or podman containers. An error in the startup configuration for the nova_libvirt container results in the libvirt virtlogd daemon not being run, which results in this console log rotation not occurring. This opens up the risk of a disk usage denial of service attack on a hypervisor. The console log is stored on the same filesystem as ephemeral disks, and OpenStack Nova will take a hypervisor out of service if disk space becomes constrained. There is some protection from the instance console not being able to write data particularly quickly, but it is definitely still possible to fill the hypervisor file system -- especially if multiple instances are used. This misconfiguration appears to exist in all versions of Kolla / Kolla-Ansible as best as I can tell. Flip qemu.conf's stdio_handler from "file" to "logd" so libvirt routes instance console chardevs through virtlogd, which rotates them per /etc/libvirt/virtlogd.conf. Without this, virtlogd runs but is never consulted and console.log grows unbounded. The nova_libvirt command now points at the /usr/local/bin/kolla_nova_libvirt_start helper shipped in the image. Deploying this Kolla-Ansible change against an older image that predates that helper would leave the container unable to start. Add a deploy-time check that inspects the nova_libvirt image for the helper and fails with an explanatory message when it is absent, so an incomplete upgrade (new code, stale image) is caught before the container is reconfigured. Partial-Bug: 2155214 Depends-On: https://review.opendev.org/c/openstack/kolla/+/993065 Change-Id: Ia543939b17cd06714695dff8ef8bd29bc3f8175e (cherry picked from commit 4244e6a91767a4706e59145f6841adbeeac048bb) Signed-off-by: Michael Still <mikal@stillhq.com>
2026-06-23Merge "Fix stray trailing quote in barbican_api healthcheck" into stable/2026.1Zuul1-1/+1
2026-06-22Fix stray trailing quote in barbican_api healthcheckPiotr Milewski1-1/+1
Remove a stray trailing quote from the barbican_api healthcheck URL. The quote causes shell parsing failures, incorrectly marking the container as unhealthy. Closes-Bug: #2157639 Change-Id: I3e9b791d0ee2ce54af862f3e30e37e6600a350bd Signed-off-by: Piotr Milewski <vurmil@gmail.com> (cherry picked from commit e101828d1887522efcd1cf7fae3606efc05f513a)
2026-06-22fix kolla_toolbox running with wrong HOME envBertrand Lanson1-1/+4
There is a possibility that the ansible-runner in kolla_toolbox is executed with the wrong environment variables if run in a specific way. The cause is not super clear, but the solution is to simply force it to use the correct HOME value. Closes-Bug: #2153544 Change-Id: Ibad84e358f21564c570161f7e4f21b117398df3b Signed-off-by: Bertrand Lanson <bertrand.lanson@infomaniak.com> (cherry picked from commit 2104f74a2616821c2bd848411ebfce266a3edd69)
2026-06-18Set Ironic tempdirTBExtent2-0/+9
Set the Ironic tempdir to be within the Ironic Docker container volume. Closes-Bug: #2156758 Change-Id: Ib24a04657f297ebb323686aa1d80f9bf019f44f2 Signed-off-by: Jed Preist <jed@stackhpc.com> (cherry picked from commit ba5d4219749099cb7a797deb77685c5293fb9bf5)
2026-06-17Fix OVN SB relay port calculation for relay counts > 9amir581184-4/+4
The OVN SB relay port was generated using string concatenation between the port prefix and relay group ID. When the relay count exceeded 9, this resulted in invalid port numbers such as 166410. Update the port calculation to use numeric addition instead, ensuring ports remain within the expected and valid range. Closes-Bug: #2131191 Change-Id: I4738339431483c3d38fb322c18888fe61cae1a62 Signed-off-by: amir58118 <sahahmadi96@gmail.com> Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com> (cherry picked from commit 6c2b9225d52cf2470d7ed85fa5fde833d764bb3c)
2026-06-09proxysql: fix keepalived healthcheck to use MySQL protocolMichal Nasiadka1-1/+7
The check_alive_proxysql.sh script piped raw ASCII text ("show info\n") directly into the admin socket via socat, bypassing the MySQL handshake. ProxySQL's MySQL_Data_Stream interprets the first 3 bytes of "show" as a little-endian packet length (7301235), which has always been semantically wrong - the check only appeared to work because socat received ProxySQL's binary handshake greeting back and exited 0. ProxySQL 3.0.9 introduced a pre-authentication bounds check (GHSA-58ww-865x-grpr) that explicitly rejects first packets whose declared length exceeds the 32 KB input queue capacity and closes the connection. This causes socat to exit non-zero, making keepalived mark ProxySQL as unhealthy on every check interval. Replace the raw socat probe with a mariadb client connection that speaks proper MySQL protocol against the admin socket. Closes-Bug: #2155865 Depends-On: https://review.opendev.org/c/openstack/kolla/+/992394 Change-Id: Ic090916509cbb1bd98c96e8beb32ec9e90bfc6aa Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com> (cherry picked from commit 1e0fa6c62bbb6129f29f01605a00ce3e0f50e27a)
2026-05-28Merge "Fix Skyline nginx service proxy paths" into stable/2026.1Zuul1-17/+17
2026-05-25CI: Fix HAProxy template for new Jinja lint rulesBartosz Bezak1-2/+1
Change-Id: Ib82e708137679ae85f645c448cc8aa3cf583cf6d Signed-off-by: Bartosz Bezak <bartosz@stackhpc.com> (cherry picked from commit 6b2112f0bc8f99d31fb7f447728af8f55db3596d)
2026-05-15Fix Skyline nginx service proxy pathsIlia Petrov1-17/+17
Skyline nginx service locations are defined without a trailing slash while proxy_pass includes a trailing slash. This can produce paths with a double leading slash, for example //resource_providers for Placement. Some WSGI deployments may treat such paths literally and return errors instead of normalizing them. Closes-Bug: #2151770 Change-Id: I530cefc1ca96d315c8be9b2e1337a09c349b9eb5 Signed-off-by: Ilia Petrov <il.petrov@maxima-int.com> (cherry picked from commit 3bd2786072c1c8f4e078a6bd420483f6732697f9)
2026-05-12Fix for overly restrictive permissions on ca-certificates directoryRafal Lewandowski135-135/+135
Closes-Bug: #2112280 Change-Id: I52499bdf2b7fc70f91a4c2920749bfef83a79850 Signed-off-by: Rafal Lewandowski <rafal@stackhpc.com> Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com> (cherry picked from commit e302bab31f30296f7d6f41c3d46e19cf5668b97a)
2026-05-11Switch openstack_release to 2026.1Michal Nasiadka1-1/+1
Change-Id: Icf7b5877f6865191f7b80d903608a6f7d5962527 Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
2026-05-06Merge "rabbitmq: fix reset-state on RabbitMQ 4.x"Zuul1-1/+6
2026-05-06Merge "haproxy: fix missing user in rabbitmq healthcheck"Zuul1-1/+1
2026-05-06Merge "opensearch: automate index pattern creation in dashboards"Zuul1-0/+27
2026-05-06Merge "prechecks: Add a precheck if using test images (quay.io/openstack.kolla)"Zuul3-0/+14
2026-05-06rabbitmq: fix reset-state on RabbitMQ 4.xPiotr Milewski1-1/+6
kolla-ansible rabbitmq-reset-state fails because rabbitmqctl force_reset is not supported with Khepri. Use rabbitmqctl reset instead. Closes-Bug: #2150536 Change-Id: Idc275dc830838519e40a519111ae0e5ca2b76edf Signed-off-by: Piotr Milewski <vurmil@gmail.com>
2026-05-06Merge "kolla_container: Return annotation of comparison failures"Zuul3-35/+183
2026-05-06kolla_container: Return annotation of comparison failuresMichal Nasiadka3-35/+183
Assisted-By: Claude Sonnet 4.6 Change-Id: If6f09c727bc5c7af27786cdc92c998d653dd4ad1 Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
2026-05-06Merge "letsencrypt: Switch webserver image to httpd"Zuul2-1/+9
2026-05-05Merge "keystone: remove use_stderr to avoid duplicate logs"Zuul1-1/+0
2026-05-05prechecks: Add a precheck if using test images (quay.io/openstack.kolla)Michal Nasiadka3-0/+14
Change-Id: I2babda3d1388f1ee825fa79a1d85d3113b5ecbd4 Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
2026-05-05Merge "ironic: Switch ironic-http to use httpd image"Zuul1-1/+5
2026-05-04Merge "Remove [fwaas] agent_version"Zuul1-1/+0
2026-04-30Remove [fwaas] agent_versionTakashi Kajinami1-1/+0
The option was deprecated during 2025.2 cycle because it has had no effect, and will be removed in 2026.2 release. [1] https://review.opendev.org/c/openstack/neutron-fwaas/+/953212 Change-Id: I4513f471d9457c1a075e9dd32720507736349bec Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
2026-04-28haproxy: fix missing user in rabbitmq healthcheckPiotr Milewski1-1/+1
RabbitMQ HAProxy config was missing user/auth in health checks, causing backend to be marked DOWN and HTTP 500 errors. Closes-Bug: #2150569 Change-Id: I1949210687bbe8f896e6b973950ddbea03c1f804 Signed-off-by: Piotr Milewski <vurmil@gmail.com>
2026-04-28Merge "nova-compute: Copy multipath.conf into nova-compute container"Zuul3-0/+34
2026-04-28ironic: Switch ironic-http to use httpd imageMichal Nasiadka1-1/+5
Change-Id: If655cd51857b65b16bde1f8870edbd217f5c10e5 Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
2026-04-27letsencrypt: Switch webserver image to httpdMichal Nasiadka2-1/+9
Switch to httpd image which was introduced for keystone-httpd and is identical to letsencrypt-webserver image. The letsencrypt-webserver image will be removed afterwards in [1]. [1]: https://review.opendev.org/c/openstack/kolla/+/986151 Depends-On: https://review.opendev.org/c/openstack/kolla/+/986377 Change-Id: Ica381670de4b157c97be4089eaeb7fe2f7ba039f Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
2026-04-27opensearch: automate index pattern creation in dashboardsPiotr Milewski1-0/+27
Previously, operators had to manually create the default index pattern after the first login to OpenSearch Dashboards. This commit automates the creation of the default index pattern using the OpenSearch Dashboards Saved Objects API during the post-config phase. The pattern is dynamically named based on the opensearch_log_index_prefix variable (defaulting to "flog-*") and automatically sets "@timestamp" as the time field. Documentation has been updated to reflect that the first-login manual setup is no longer required for the default log indices. Change-Id: I8d49a270ace187b3b13103452e93d305505f6cbb Signed-off-by: Piotr Milewski <vurmil@gmail.com>
2026-04-27Merge "Drop zun and kuryr"Zuul63-1489/+1
2026-04-27Merge "manila: Fix manila-api.log location"Zuul1-0/+3
2026-04-27Merge "Make RabbitMQ stream retention policy configurable"Zuul3-1/+52
2026-04-27Merge "Update name of dataplane node exporter"Zuul1-2/+2
2026-04-27manila: Fix manila-api.log locationMichal Nasiadka1-0/+3
Without that it ends up as .log or some other Python frozenset.log Change-Id: I3a6e2541aab07f822402f0b5fca96f8aba7e2f4a Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
2026-04-27Drop zun and kuryrMichal Nasiadka63-1489/+1
Zun is broken in 2026.1, it has been released broken and there are no reviews on the patch to fix it [1]. [1]: https://review.opendev.org/c/openstack/zun/+/982362 Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/986150 Change-Id: I8ced621754d03c44b7672dd94fc3d7d2173ac734 Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
2026-04-25Merge "Support multiple Nova Compute Ironic instances"Zuul16-12/+274
2026-04-25keystone: remove use_stderr to avoid duplicate logsMichal Arbet1-1/+0
use_stderr may cause log duplication when stderr is captured by uWSGI. Remove it so logs are written only to log_file. Closes-Bug: #2148545 Change-Id: Ia552aa2fad26609d7e2fe6a726db785fb2a7f635 Signed-off-by: Michal Arbet <michal.arbet@ultimum.io>
2026-04-24Merge "Drop support for running WSGI under Apache"Zuul72-1461/+105
2026-04-24Support multiple Nova Compute Ironic instancesDoug Szumski16-12/+274
Adds support for deploying multiple instances of the Nova Compute Ironic service on the same host. This is useful in large baremetal deployments, where the sharding and/or conductor group feature is used to scale out the service. A further patch to support deploying multiple Ironic conductor instances will follow. Co-Authored-By: Bartosz Bezak <bartosz@stackhpc.com> Co-Authored-By: Bertrand Lanson <bertrand.lanson@protonmail.com> Change-Id: Ibbddfd87e831a3775c3ed66c80b18e5070cedc90 Signed-off-by: Doug Szumski <doug@stackhpc.com>
generated by cgit v1.2.3 (git 2.39.1) at 2026-11-05 16:06:44 +0000