1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
|
.. troubleshooting:
=====================
Troubleshooting Guide
=====================
Failures
~~~~~~~~
If Kolla fails, often it is caused by a CTRL-C during the deployment
process or a problem in the ``globals.yml`` configuration.
To correct the problem where Operators have a misconfigured environment,
the Kolla community has added a precheck feature which ensures the
deployment targets are in a state where Kolla may deploy to them. To
run the prechecks:
.. code-block:: console
kolla-ansible prechecks
If a failure during deployment occurs it nearly always occurs during evaluation
of the software. Once the Operator learns the few configuration options
required, it is highly unlikely they will experience a failure in deployment.
Deployment may be run as many times as desired, but if a failure in a
bootstrap task occurs, a further deploy action will not correct the problem.
In this scenario, Kolla's behavior is undefined.
The fastest way during to recover from a deployment failure is to
remove the failed deployment:
.. code-block:: console
kolla-ansible destroy -i <<inventory-file>>
Any time the tags of a release change, it is possible that the container
implementation from older versions won't match the Ansible playbooks in a new
version. If running multinode from a registry, each node's Docker image cache
must be refreshed with the latest images before a new deployment can occur. To
refresh the docker cache from the local Docker registry:
.. code-block:: console
kolla-ansible pull
Debugging Kolla
~~~~~~~~~~~~~~~
The status of containers after deployment can be determined on the deployment
targets by executing (use `podman` instead of `docker` if applicable):
.. code-block:: console
docker ps -a
If any of the containers exited, this indicates a bug in the container. Please
seek help by filing a `launchpad bug <https://bugs.launchpad.net/kolla-ansible/+filebug>`__
or contacting the developers via IRC.
The logs can be examined by executing:
.. code-block:: console
docker exec -it fluentd bash
The logs from all services in all containers may be read from
``/var/log/kolla/SERVICE_NAME``
If the stdout logs are needed, please run:
.. code-block:: console
docker logs <container-name>
Note that most of the containers don't log to stdout so the above command will
provide no information.
To learn more about container engine command line operation, please refer
to the `Docker
documentation <https://docs.docker.com/reference/>`__ or the `Podman
documentation <https://docs.podman.io/en/stable/Commands.html>`__.
The log volume "kolla_logs" is linked to ``/var/log/kolla`` on the host.
You can find all kolla logs in there.
.. code-block:: console
readlink -f /var/log/kolla
/var/lib/docker/volumes/kolla_logs/_data
When ``enable_central_logging`` is enabled, to view the logs in a web browser
using OpenSearch Dashboards, go to
``http://<kolla_internal_vip_address>:<opensearch_dashboards_port>`` or
``http://<kolla_external_vip_address>:<opensearch_dashboards_port>``. Authenticate
using ``opensearch`` and ``<opensearch_dashboards_password>``.
The values ``<kolla_internal_vip_address>``, ``<kolla_external_vip_address>``
``<opensearch_dashboards_port>`` can be found in
``<kolla_install_path>/kolla/ansible/group_vars/all/opensearch.yml``. The value
of ``<opensearch_dashboards_password>`` can be found in
``/etc/kolla/passwords.yml``.
|
