| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Also add all failed roles into ignore and these will be removed in
subsequent patches
Change-Id: Ifa6f2725cfa93cb14d72daf27448bd58f186acfd
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
Normalize quotes and some other easy fixes done by
ansible-lint --fix
Also drop etc from excludes and ignore comment check in
etc/kolla/globals.yml
Pinning to ansible-lint<25.11.0 because 25.11.1 is breaking
without any messages
Change-Id: I7dee3866191cef84cd12b1816cdab0a02380ec7f
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
|
|
When Kolla Ansible is executed with a '--limit' argument, the scope of
an operation is limited to the hosts in the limit. For example:
kolla-ansible deploy --limit control
Due to the nature of configuring clustered software services, there are
cases where we need to know information about other hosts. Most often
this is related to their hostname or network addresses. To make this
work, Kolla Ansible gathers facts for hosts outside of the limit using
delegated fact gathering [1].
By default, Kolla Ansible gathers facts for all hosts. Because delegated
facts are gathered serially in batches by the active hosts, this can
take a long time when there are not many hosts in the limit.
This change makes it possible to reduce the set of hosts eligible for
delegated fact gathering by setting 'kolla_ansible_delegate_facts_hosts'
to a list of hosts.
[1] https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_delegation.html#delegating-facts
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/899615
Change-Id: Id0bda00b81b5bc6a9a870e231335c33d9828e95f
|
|
|
|
|
|
|
|
|
|
This reverts commit 5b431f0f7f932293362a01703673ee05d0a5bd8d.
Reason for revert: the any_errors_fatal play parameter is not templated
by Ansible (tested up to ansible-core 2.15.9). This behaviour is
demonstrated in [1].
This means that "{{ kolla_ansible_setup_any_errors_fatal }}" is always
interpreted as 'true', regardless of the value of
kolla_ansible_setup_any_errors_fatal. This is particularly bad because
the default value of kolla_ansible_setup_any_errors_fatal is false.
We now have gather_facts_max_fail_percentage which can be set to 0 to
provide the same functionality.
[1] https://github.com/markgoddard/ansible-experiments/tree/master/15-fatal-errors
Change-Id: I2e0ea49701b5900eae26434bcdb6b1bb44507ee7
|
|
Previously, during fact gathering when using the '--limit' argument if a
host inside the limit fails to gather facts, this would cause another
host to fail during delegated fact gathering. This is because the
ansible_facts variable would not be populated for the host.
This change fixes the issue by avoiding delegated fact gathering for
hosts inside the limit.
Partial-Bug: #2041860
Change-Id: I8e321469089f0eb80420b788b51e6d028c7fef4d
|
|
This allows us to continue execution until a certain proportion of hosts
to fail. This can be useful at scale, where failures are common, and
restarting a deployment is time-consuming.
The default max failure percentage is 100, keeping the default
behaviour. A global max failure percentage may be set via
kolla_max_fail_percentage, and individual services may define a max
failure percentage via <service>_max_fail_percentage.
Note that all hosts in the inventory must be reachable for fact
gathering, even those not included in a --limit.
Closes-Bug: #1833737
Change-Id: I808474a75c0f0e8b539dc0421374b06cea44be4f
|
|
If not using --limit, each host runs the setup module for itself only.
Hosts should be able to fail at this point and drop out of execution
without affecting others. However, there is a bug that causes us to hit
the --limit code path for an unreachable host and therefore become
subject to its limitations. These include the use of delegated fact
gathering, and failing the delegated host when the delegating host is
unreachable.
This change fixes the issue by reversing the order of two tasks in the
gather-facts.yml playbook, ensuring that we check the ansible_play_batch
variable before any hosts have had a chance to fail.
Change-Id: I4b1da63e8f0cc2774f1a9ab2e0414746108f7e12
Closes-Bug: #2041859
|
|
These were added in I20e65a6771ebeee462a3aaaabaa5f0596bdd0581 but not
used in the final version of the changeset.
Change-Id: I4fca47e16e87db8b655ff5571fba568ef4edb95c
|
|
This commit adds SystemdWorker class to kolla_docker ansible module.
It is used to manage container state via systemd calls.
Change-Id: I20e65a6771ebeee462a3aaaabaa5f0596bdd0581
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
|
|
Kolla Ansible now supports failing execution early if fact collection
fails on any of the hosts. This is to avoid late failures due to missing
facts (especially cross-host).
Change-Id: I7a74b937ded0b9da0621cf413f3a5d0d13a2cd68
Partial-Bug: #1833737
|
|
Ansible facts can have a large impact on the performance of the Ansible
control host. This patch introduces some control over which facts are
gathered (kolla_ansible_setup_gather_subset) and which facts are stored
(kolla_ansible_setup_filter). By default we do not change the default
values of these arguments to the setup module. The flexibility of these
arguments is limited, but they do provide enough for a large performance
improvement in a typical moderate to large OpenStack cloud.
In particular, the large complex dict fact for each interface has a
large effect, and on an OpenStack controller or hypervisor there may be
many virtual interfaces. We can use the kolla_ansible_setup_filter
variable to help:
kolla_ansible_setup_filter: 'ansible_[!qt]*'
This causes Ansible to collect but not store facts matching that
pattern, which includes the virtual interface facts. Currently we are
not referencing other facts matching the pattern within Kolla Ansible.
Note that including the 'ansible_' prefix causes meta facts module_setup
and gather_subset to be filtered, but this seems to be the only way to
get a good match on the interface facts. To work around this, we use
ansible_facts rather than module_setup to detect whether facts exist in
the cache.
The exact improvement will vary, but has been reported to be as large as
18x on systems with many virtual interfaces.
For reference, here are some other tunings tried:
* Increased the number of forks (great speedup depending of the size of
the deployment)
* Use `strategy = mitogen_linear` (cut processing time in half)
* Ansible caching (little speed up)
* SSH tunning (little speed up)
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
Closes-Bug: #1921538
Change-Id: Iae8ca4aae945892f1dc65e1b10381d2e26e88805
|
|
By default, Ansible injects a variable for every fact, prefixed with
ansible_. This can result in a large number of variables for each host,
which at scale can incur a performance penalty. Ansible provides a
configuration option [0] that can be set to False to prevent this
injection of facts. In this case, facts should be referenced via
ansible_facts.<fact>.
This change updates all references to Ansible facts within Kolla Ansible
from using individual fact variables to using the items in the
ansible_facts dictionary. This allows users to disable fact variable
injection in their Ansible configuration, which may provide some
performance improvement.
This change disables fact variable injection in the ansible
configuration used in CI, to catch any attempts to use the injected
variables.
[0] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#inject-facts-as-vars
Change-Id: I7e9d5c9b8b9164d4aee3abb4e37c8f28d98ff5d1
Partially-Implements: blueprint performance-improvements
|
|
This trivial patch is just turning off ansible
changed report for group_by tasks as it could
be confusing for user.
Change-Id: I7512af573782359a6f01290a55291ac7eb0de867
|
|
One way to improve the performance of Ansible is through fact caching.
Rather than gather facts in every play, we can configure Ansible to
cache them in a persistent store. An example Ansible configuration for
doing this is as follows:
[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./facts
fact_caching_timeout = 86400
This does not affect Kolla Ansible however, since we use the setup
module which unconditionally gathers facts regardless of the state of
the cache. This gets worse with large inventories limited to a small
batch of hosts via --limit or serial, since the limited hosts must
gather facts for all others.
One way to detect whether facts exist for a host is via the
'module_setup' variable, which exists only when facts exist. This change
uses the 'module_setup' fact to determine whether facts need to be
gathered for hosts outside of the batch. For hosts in the batch, we
switch from using the setup module to gather_facts on the play, which
can use the 'smart' gathering logic.
Change-Id: I04841fb62b2e1d9e97ce4b75ce3a7349b9c74036
Partially-Implements: blueprint performance-improvements
|
|
Prior to this change, when the --limit argument is used, each host in the
limit gathers facts for every other host. This is clearly unnecessary, and
can result in up to (N-1)^2 fact gathers.
This change gathers facts for each host only once. Hosts that are not in
the limit are divided between those that are in the limit, and facts are
gathered via delegation.
This change also factors out the fact gathering logic into a separate
playbook that is imported where necessary.
Change-Id: I923df5af41a7f1b7b0142d0da185a9a0979be543
|
