| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Closes-Bug: #2112280
Change-Id: I52499bdf2b7fc70f91a4c2920749bfef83a79850
Signed-off-by: Rafal Lewandowski <rafal@stackhpc.com>
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
(cherry picked from commit e302bab31f30296f7d6f41c3d46e19cf5668b97a)
|
|
Change-Id: I8a39009a31ba6ea966a8dc577187620d77d16b99
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
Since ipwrap filter is getting moved another time
inside ansible code - let's switch to j2lint that is
actively maintained and it's rules will help in
readability of jinja2 templates
Change-Id: I6762b691b927de31fa1abd83c4c14e87226fe5e2
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
Remove /etc/timezone bind mount since Debian does not provide
that file anymore.
Depends-On: https://review.opendev.org/c/openstack/kolla/+/964793
Change-Id: If1a289068093843184f0d7b495815db006f2d563
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
Change-Id: Iba6f15faec8144072563f44ba2e4b6f86464785a
Signed-off-by: Bertrand Lanson <bertrand.lanson@protonmail.com>
|
|
Change-Id: I7c693403b1768c6c97eea90c3dfe9326764c3230
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
Change-Id: I719be4c046c0bb5153a616621259909bce6964c3
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
Bumped collections in requirements-core.yml to Ansible 12 major versions,
see [1].
[1]: https://github.com/ansible-community/ansible-build-data/blob/main/12/ansible-12.1.0.yaml
Depends-On: https://review.opendev.org/c/openstack/kolla/+/960307
Co-authored-by: Doug Szumski doug@stackhpc.com
Change-Id: If8a29052d8a43ffc0fef11514adabb5e4ab9d977
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
ceilometer-polling is not connecting to RMQ on empty deployments, let's
disable healthchecks for now until we find a better healthcheck.
Closes-Bug: #1992515
Change-Id: I6c6202daa3e29ed41c859e34cbbe91adc6660e36
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
Change-Id: I20a669bf7796285df04723a9110f9c74788600e1
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
Since Kolla-Ansible now supports TLS for MariaDB,
this patch adds support for services that can use
this feature.
This feature can be used only when ProxySQL is used
as LB.
Signed-off-by: Michal Arbet <michal.arbet@ultimum.io>
Change-Id: I052c763073c5da29cb8e78004a01a22217dde5da
|
|
Stop repeating the same code (actually 158 times).
Change-Id: I2fd04e6ed72f3702eb41fe5cf172f47a33e93bbd
|
|
We will be upgrading RabbitMQ to version 4.0 in Epoxy. This will not
work without quorum queues being enabled.
Change-Id: Ic6ad64bf8c62bbff175e15029eb121814032c40e
|
|
We're going to upgrade RabbitMQ to 4.0, so this option will not longer
be supported.
Change-Id: Ide75a8c9086798bf4bdf5bc02d4a1be17017884f
|
|
The global configuration of rabbit_qos_prefetch_count = 1
in oslo.messaging is a deliberate choice that balances
compatibility, fairness, and reliability across all
OpenStack services.
While some services (particularly those using quorum queues)
could theoretically benefit from a higher or even unlimited
prefetch count for performance reasons, this is not universally
safe. Specifically, RabbitMQ stream queues - which are used in
certain OpenStack components do not support a prefetch_count of
0. Setting this value to 0 would result in runtime errors
or stalls, as the stream protocol requires a positive credit
window to function properly. Oslo.messaging enforces this by
raising an error if prefetch_count = 0 is set while using stream
queues.
A value of 1 is the lowest legal and universally compatible
setting. It works safely with both stream and quorum queues.
It also improves fairness in message distribution across worker
processes. In distributed services like Nova, Neutron, or
Cinder, this helps avoid uneven load where one worker might
prefetch many messages while others remain idle.
A low prefetch count ensures that messages are pulled only
when the worker is ready, promoting better load balancing and
more predictable performance.
Although a higher prefetch value could increase throughput in
certain scenarios, it comes at the cost of memory usage, risk
of overloading specific nodes, and potential starvation of
others. Until oslo.messaging supports queue-type-specific or
per-service tuning, the value of 1 remains the safest and
most predictable option that works well in mixed OpenStack
environments.
In summary, rabbit_qos_prefetch_count = 1 is not about
optimizing raw throughput for any one service, but about
ensuring stable, fair, and reliable behavior across all
services that rely on oslo.messaging, regardless of the queue
type or broker backend used.
Change-Id: I541f704bfa2e98068096331afbdb591659cbc40b
|
|
This helps to improve the reliability of openstack services when
a rabbitmq node has issues.
See also: https://bugs.launchpad.net/oslo.messaging/+bug/2031497
Adjust the upgrade tests similar to what was done
during the introduction of quorum queues in I6c033d460a5c9b93c346e9e47e93b159d3c27830
Closes-Bug: #2078339
Partial-Bug: #2077448
Depends-On: https://review.opendev.org/c/openstack/oslo.messaging/+/888479
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/924623
Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: Idb8a8d2e560206f7697c0771c9ae3913268fa6dd
|
|
|
|
Adds the variable ``om_enable_queue_manager`` to configure enabling
oslo.messaging Queue Manager for all services which use RabbitMQ. This
is enabled by default.
This is required before we can move away from transient queues
(``rabbit_transient_quorum_queue``) as Queue Manager is needed to avoid
consuming all erlang atoms after some time. It is also a useful feature
for debugging, as queues are now named with hostnames and services
included.
Also setting ``lock_path`` and mounting ``dev/shm`` is dependent on
``om_enable_queue_manager``, these are now enabled too. This will allow
us to backport these features to Caracal and Dalamation without
enforcing the changes right away.
Why is -t added for exec commands?
When using Podman, behavior differs from Docker regarding exec
commands. If -t (which allocates a pseudo-terminal) is not used, the
process runs as a child of podman exec without a PTY. This causes
Python inside the container to assign PGID=0 because the pseudo-terminal
is missing, leading to issues with non-existent paths like /proc/0/....
Adding -t ensures proper PGID assignment, preventing these issues and
ensuring consistent behavior across Docker and Podman.
Side-notes:
The quorum queue precheck needed to be updated as queue manager puts
"_fanout" on the end of the name instead.
We need to override the ``[oslo_messaging_rabbit] processname`` for
services running udner wsgi, as they will otherwise all use the same
processname ``mod_wsgi``. This will cause Permission Errors trying to
access the same file in shared memory as services run with different
users.
Change-Id: Iae5f268e778fbbd2b744dc71a84253ec9e758a99
|
|
Since Swift is broken and since deprecation nobody did
pick up the work to make it working - let's remove
swift role and associated integrations/CI scripts.
Change-Id: I08e92aaeea644053fd25f80ce1f276a495cebbfc
|
|
In the past, we agreed to use lowercase letters
for boolean representation. This patch implements
exactly that, just changing True -> true, False -> false.
Change-Id: Ia601d84283dd8bc45e56e1029106de8836139173
|
|
This patch removes the conditional check
`om_enable_queue_manager` for `oslo_concurrency` as
it was inconsistently applied across services and
is actually unrelated to the queue manager. Simply
said somewhere the conditional was present and somewhere
it wasn't.
While `oslo.concurrency` itself does not
require a specific path, the implementation of the queue
manager expects locks to be placed under
`/var/lib/<service>/tmp`, making it necessary to define
this path explicitly. Therefore, the lock path is set
accordingly across all services, regardless of whether
the queue manager is used.
Additionally, this patch adds missing `lock_path`
configurations where they were absent to ensure uniformity.
Change-Id: I93bbaa61b2d8b5cb0d1a11783086b37a860173b6
|
|
The Oslo.messaging project implemented a nice feature
called Queue Manager. This means that instead of using
random queue names in RabbitMQ, it uses queue names composed
of hostname, process-name and some integer. For proper
functioning, the code using lockutils from oslo_concurrency
and it's failing if lock_path is not set.
This patch simply set lock_path of oslo_concurrency for
services which can potentionally use Queue Manager.
This is enabled when ``om_enable_queue_manager`` is set to ``True``.
Queue Manager will be supported in a follow-up patch.
Change-Id: Ided3b2bce03ea11fb34820fc40b4a3d694b8b44c
|
|
|
|
This allows operators quickly diagnose all containers across
all hosts by running kolla-ansible check. It returns a list
of containers that are missing, not running or in unhealthy
state for each OpenStack service.
Change-Id: I36119ccdeb264aa3de928ec2254d6ff4cc955bfb
Implements: blueprint check-containers
Co-Authored-By: Roman Krček <roman.krcek@tietoevry.com>
|
|
The Oslo.messaging project implemented a nice feature called Queue
Manager. This means that instead of using random queue names in
RabbitMQ, it uses queue names composed of hostname, process-name and a
counter. For proper functioning, the code stores some informations in
/dev/shm. This is needed to avoid creating queues with the same name.
We'd otherwise hit this where multiple services run under mod_wsgi, or
with services such as Magnum just within a single container as it needs
to create multiple "reply" queues.
This patch mounts /dev/shm in containers where oslo_messaging is used.
This is enabled when ``om_enable_queue_manager`` is set to ``True``.
Queue Manager will be supported in a follow-up patch.
Change-Id: Ib85ce252374fae917d329e1824800a288c6bc9f1
|
|
This might fix some hidden bugs where the check tasks forgot to
include params important for the service.
We also get a nice optimisation by using a filtered loop instead
of task skipping per service with 'when'. As proven in
https://review.opendev.org/c/openstack/kolla-ansible/+/914997
This refactoring allows for further optimisation and
fixing work to proceed with much less hassle. Including getting
rid of many notify statements as the restarts are now safely handled
by check-containers. Some notifies had to stay, because of special
edge cases eg. in rolling upgrades and loadbalancer config.
One downside is we remove the little optimisation for Zun that
ignored config change for copying loopback but this is an
acceptable tradeoff considering the benefits above.
Co-Authored-By: Roman Krček <roman.krcek@tietoevry.com>
Change-Id: I855dfef33aa0f3fd1301295bb8ede3e587e7162a
Partially-Implements: blueprint performance-improvements
|
|
This patch builds upon genconfig optimisation and it takes it
further by not having genconfig ever touch the handlers!
Calling the handlers and skipping them created an unnecessary slow down
if only config was ran. It also depends on the config checking fix.
This gets us closer to the single responsibility principle -
config only generates the config, container checks only validate
whether container restart is needed.
And this also means that we will have single place were containers
are restarted, were we can fix the ansible quirk of it restarting
the whole group even when one container changed in the following patches.
The only exception is the loadbalance role. As the loadbalancer services
have their config altered by other roles registering their services
using loadbalancer-config. This is in contrast to typical roles,
which do config in one step and can then run check-containers in
the next step.
Fixes some handlers that were missing the necessary guard,
making genconfig actually able to restart some containers.
Future work:
- optimise config by doing local generation and mass rsync
- support for reloads
- unconditional restart/reload (separate action)
- make 'reconfigure' act like 'genconfig' + 'deploy-containers'
- this would avoid calling bootstrapping each time but might
be tricky as it would break current compatibility
- could call this 'reconfigure-containers' and deprecate
'reconfigure'
- fix the ansible quirk that notifies more handlers then intended
Change-Id: I0ce24043ae5486b2b55489ba40abe2b96b0991a6
Partially-Implements: blueprint performance-improvements
Co-Authored-By: Roman Krček <roman.krcek@tietoevry.com>
|
|
This is a prerequisite for patchset #745164
This fixes unwanted restarts during copying of certificates.
By removing conditional statements from role handlers in #745164,
copying certificates caused containers to restart, this is unwanted
during the genconfig process. However, if we would remove handler
notifiers from copying certificates, the container would never
restart, since from #745164, containers will restart only if any
of the files specified in config.json change. Certificates are now
copied to intermediary location inside of the container, from which
the script kolla_copy_cacerts will install them in the trust store.
Depends-on: https://review.opendev.org/c/openstack/kolla/+/926882
Change-Id: Ib89048c7e0f250182c4bf57d8c8a1b5478e9b4ab
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>
|
|
|
|
Build upon changes in kolla which change strategy of installing projects
in containers when in dev mode. This fixes problems where when package
file manifest changes, the changes were not reflected in to
devmode-enabled container.
It changes the strategy of installing projects in dev mode in containers.
Instead of bind mounting the project's git repository to the venv
of the container, the repository is bind mounted to
/dev-mode/<project_name> from which the it is installed using pip
on every startup of the container using kolla_install_projects script.
Also updates docs to reflect the changes.
Depends-On: https://review.opendev.org/c/openstack/kolla/+/925712
Closes-Bug: #1814515
Singed-off-by: Roman Krček <roman.krcek@tietoevry.com>
Change-Id: If191cd0e3fcf362ee058549a1b6c244d109b6d9a
|
|
All of them have been renamed 4 years ago [1].
https: //review.opendev.org/c/openstack/ansible-collections-openstack/+/726538
Change-Id: I23525c35e3f6e4a4a611e97e01cab97750199aef
|
|
|
|
The Kolla project supports building images with
user-defined prefixes. However, Kolla-ansible is unable
to use those images for installation.
This patch fixes that issue.
Closes-Bug: #2073541
Change-Id: Ia8140b289aa76fcd584e0e72686e3786215c5a99
|
|
Most roles are not leveraging the jinja filters available.
According to [1] filtering the list of services makes the execution
faster than skipping the tasks.
This patchset also includes some cosmetic changes to genconfig.
Individual services are now also using a jinja filter. This has
no impact on performance, just makes the tasks look cleaner.
Naming of some vars in genconfig was changed to "service" to make
the tasks more uniform as some were previously using
the service name and some were using "service".
Three metrics from the deployment were taken and those were
- overall deployment time [s]
- time spent on the specific role [s]
- CPU usage (measured with perf) [-]
Overall genconfig time went down on avg. from 209s to 195s
Time spent on the loadbalancer role went down on avg. from 27s to 23s
Time spent on the neutron role went down on avg from 102s to 95s
Time spent on the nova-cell role went down on avg. from 54s to 52s
Also the average CPUs utilized reported by perf went down
from 3.31 to 3.15.
For details of how this was measured see the comments in gerrit.
[1] - https://github.com/stackhpc/ansible-scaling/blob/master/doc/skip.md
Change-Id: Ib0f00aadb6c7022de6e8b455ac4b9b8cd6be5b1b
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>
|
|
|
|
Also rename task to "Copying over custom pipeline.yaml file" for
clarity.
Change-Id: I04e3eb9620830a15781f9bab2549b557a9d1d9cb
|
|
It's inactive and hasn't produced a 2024.1 release [1].
[1]: https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html#current-inactive-projects
Change-Id: I888963751b6e1ed080588297c2889e700431516c
|
|
This implements a global toggle `om_enable_rabbitmq_quorum_queues`
to enable quorum queues for each service in RabbitMQ, similar to
what was done for HA[0].
Quorum Queues are enabled by default.
Quorum queues are more reliable, safer, simpler and faster than
replicated mirrored classic queues[1].
Mirrored classic queues are deprecated and scheduled for removal
in RabbitMQ 4.0[2].
Notice, that we do not need a new policy in the RabbitMQ definitions
template, because their usage is enabled on the client side and can't
be set using a policy[3].
Notice also, that quorum queues are not yet enabled in oslo.messaging
for the usage of reply_ and fanout_ queues (transient queues).
This will change once[4] is merged.
[0]: https://review.opendev.org/c/openstack/kolla-ansible/+/867771
[1]: https://www.rabbitmq.com/quorum-queues.html
[2]: https://blog.rabbitmq.com/posts/2021/08/4.0-deprecation-announcements/
[3]: https://www.rabbitmq.com/quorum-queues.html#declaring
[4]: https://review.opendev.org/c/openstack/oslo.messaging/+/888479
Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: I6c033d460a5c9b93c346e9e47e93b159d3c27830
|
|
Changes name of ansible module kolla_docker to
kolla_container.
Change-Id: I13c676ed0378aa721a21a1300f6054658ad12bc7
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
|
|
docker_restart_policy: no causes systemd units to not get created
and we use it in CI to disable restarts on services.
Introducing oneshot policy to not create systemd unit for oneshot
containers (those that are running bootstrap tasks, like db
bootstrap and don't need a systemd unit), but still create systemd
units for long lived containers but with Restart=No.
Change-Id: I9e0d656f19143ec2fcad7d6d345b2c9387551604
|
|
This change adds basic deployment based on Podman
container manager as an alternative to Docker.
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
Signed-off-by: Petr Tuma <p.tuma@partner.samsung.com>
Change-Id: I2b52964906ba8b19b8b1098717b9423ab954fa3d
Depends-On: Ie4b4c1cf8fe6e7ce41eaa703b423dedcb41e3afc
|
|
Allow more flexibility by adding a host specific template. Also
using the merge_yaml action allow us to drop the logic around deciding
which template to use.
Change-Id: I41a3226679c622e5e98df074cee195e8436ea45b
|
|
Signed-off-by: FelipeAFV <ffigueroa@whitestack.com>
Closes-Bug: #2009518
Change-Id: I8c4b0053f2f16b6d243462c4b8117748d26143a0
|
|
A combination of durable queues and classic queue mirroring can be used
to provide high availability of RabbitMQ. However, these options should
only be used together, otherwise the system will become unstable. Using
the flag ``om_enable_rabbitmq_high_availability`` will either enable
both options at once, or neither of them.
There are some queues that should not be mirrored:
* ``reply`` queues (these have a single consumer and TTL policy)
* ``fanout`` queues (these have a TTL policy)
* ``amq`` queues (these are auto-delete queues, with a single consumer)
An exclusionary pattern is used in the classic mirroring policy. This
pattern is ``^(?!(amq\\.)|(.*_fanout_)|(reply_)).*``
Change-Id: I51c8023b260eb40b2eaa91bd276b46890c215c25
|
|
|
|
The ``[oslo_messaging_rabbit] heartbeat_in_pthread`` config option
is set to ``true`` for wsgi applications to allow the RabbitMQ
heartbeats to function. For non-wsgi applications it is set to ``false``
as it may otherwise break the service [1].
[1] https://docs.openstack.org/releasenotes/oslo.messaging/zed.html#upgrade-notes
Change-Id: Id89bd6158aff42d59040674308a8672c358ccb3c
|
|
Regularly, we experience issues in Kolla Ansible deployments because we
use wrong options in OpenStack configuration files. This is because
OpenStack services ignore unknown options. We also need to keep on top
of deprecated options that may be removed in the future. Integrating
oslo-config-validator into Kolla Ansible will greatly help.
Adds a shared role to run oslo-config-validator on each service. Takes
into account that services have multiple containers, and these may also
use multiple config files. Service roles are extended to use this shared
role. Executed with the new command ``kolla-ansible validate-config``.
Change-Id: Ic10b410fc115646d96d2ce39d9618e7c46cb3fbc
|
|
Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/
in which was suggested to split patch into smaller ones.
THis change adds container_engine to module parameters
so when we introduce podman, kolla_toolbox can be used
for both engines.
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: Ic2093aa9341a0cb36df8f340cf290d62437504ad
|
|
ansible-lint introduced var-spacing - let's fix our code.
Change-Id: I0d8aaf3c522a5a6a5495032f6dbed8a2be0251f0
|
|
Per comments on [1].
[1] https://review.opendev.org/c/openstack/kolla-ansible/+/843727
Change-Id: I60162b54bc06e158534d29311d4474b34750c64d
|
