Change Bifrost Python upper-constraints source

Users can override the upper-constraints.txt by setting the source of
openstack-base in kolla-build.conf.

However, bifrost-base Dockerfile so far doesn't pass overridden
upper-constraints file to Bifrost install playbook resulting the
playbook not acknowledging it and just use upstream one.

This change sets the source of Bifrost upper-constraints to use
upper-constraints of openstack-base for consistency among all Kolla
images

With this, change the environment variable UPPER_CONSTRAINTS_FILE is
only affecting kolla-toolbox image.
Therefore, moved UPPER_CONSTRATINS_FILE variable example to the
``kolla_toolbox_pip_conf`` block of kolla-toolbox Dockerfile.

Additionally, updated 'Python packages build options' section of
documentation to represent latest method of changing Python package
related options (including this change).

Change-Id: I0a9c782ca1177ef84e4323cb1e5bfac0fb3f383c

5 files changed, 83 insertions, 7 deletions

diff --git a/doc/source/admin/image-building.rst b/doc/source/admin/image-building.rst
index aad3c33a3..02abb19b9 100644
--- a/doc/source/admin/image-building.rst
+++ b/doc/source/admin/image-building.rst
@@ -320,11 +320,69 @@ Python packages build options
 
 The block ``base_pip_conf`` in the ``base`` Dockerfile can be used to provide
 the PyPI build customisation options via the standard environment variables
-like ``PIP_INDEX_URL``, ``PIP_TRUSTED_HOST``, etc. Also here can be provided
-the standard environment variable ``UPPER_CONSTRAINTS_FILE`` used for building
-the ``bifrost_deploy`` container when PyPI upper-constraints needs to be
-overridden. Also this variable would be used in the ``kolla-toolbox`` if
-provided instead of the defaults.
+like ``PIP_INDEX_URL``, ``PIP_TRUSTED_HOST``, etc.
+
+To override PYPI upper-constraints of all OpenStack images, you can
+define the source location of openstack-base. in ``kolla-build.conf``.
+
+Upstream repository of `openstack-base (requirements) <https://opendev.org/openstack/requirements>`__
+has a source of
+`upper constraints file <https://opendev.org/openstack/requirements/src/branch/master/upper-constraints.txt>`__.
+
+Make a fork or clone the repository then customise ``upper-constraints.txt``
+and define the location of ``openstack-base`` in ``kolla_build.conf``.
+
+.. path /etc/kolla/kolla-build.conf
+.. code-block:: ini
+
+   # These examples use upstream openstack-base as a demonstration
+   # To use custom openstack-base, make changes accordingly
+
+   # Using git source
+   [openstack-base]
+   type = git
+   location = https://opendev.org/openstack/requirements
+   reference = master
+
+   # Using URL source
+   [openstack-base]
+   type = url
+   location = https://tarballs.opendev.org/openstack/requirements/requirements-master.tar.gz
+
+   # Using local source
+   [openstack-base]
+   type = local
+   location = /home/kolla/src/requirements
+
+To remove or change the version of specific Python packages in
+``openstack-base`` upper-constraints, you can use the block
+``openstack_base_override_upper_constraints`` in your template file,
+for example, ``template-overrides.j2``:
+
+.. code-block:: jinja
+
+   {% block openstack_base_override_upper_constraints %}
+   RUN {{ macros.upper_constraints_version_change("sqlparse", "0.4.4", "0.5.0") }}
+   RUN {{ macros.upper_constraints_remove("reno") }}
+   {% endblock %}
+
+``kolla-toolbox`` image needs different approach as it does not uses
+``openstack-base`` as a base image.
+A variable ``UPPER_CONSTRAINTS_FILE`` is set in the
+Dockerfile of ``kolla-toolbox``.
+To change variable, add the following contents to the
+``kolla_toolbox_pip_conf`` block in your template file, for example,
+``template-overrides.j2``:
+
+.. code-block:: jinja
+
+   {% block kolla_toolbox_pip_conf %}
+   ENV UPPER_CONSTRAINTS_FILE=https://releases.openstack.org/constraints/upper/master
+   {% endblock %}
+
+.. note::
+
+   ``UPPER_CONSTRAINTS_FILE`` must be a valid URL to the file
 
 Plugin functionality
 --------------------
diff --git a/docker/base/Dockerfile.j2 b/docker/base/Dockerfile.j2
index bb61c8307..c7ec3de1f 100644
--- a/docker/base/Dockerfile.j2
+++ b/docker/base/Dockerfile.j2
@@ -379,7 +379,6 @@ RUN touch /usr/local/bin/kolla_extend_start \
 # the variables like PIP_INDEX_URL, PIP_EXTRA_INDEX_URL, PIP_TRUSTED_HOST etc. should be defined here.
 # ENV PIP_INDEX_URL=https://pypi.python.org/simple
 # ENV PIP_TRUSTED_HOST=pypi.python.org
-# ENV UPPER_CONSTRAINTS_FILE=https://releases.openstack.org/constraints/upper/{{ openstack_release }}
 {% endblock %}
 
 {% block base_footer %}{% endblock %}
diff --git a/docker/bifrost/bifrost-base/Dockerfile.j2 b/docker/bifrost/bifrost-base/Dockerfile.j2
index 3d0fe726f..8aa8f54ae 100644
--- a/docker/bifrost/bifrost-base/Dockerfile.j2
+++ b/docker/bifrost/bifrost-base/Dockerfile.j2
@@ -45,6 +45,7 @@ RUN echo " " && \
     $VENV/bin/ansible-galaxy collection install -r /bifrost/ansible-collections-requirements.yml && \
     $VENV/bin/ansible-playbook -vvvv -i /bifrost/playbooks/inventory/target \
     /bifrost/playbooks/install.yaml \
+    -e upper_constraints_file="/requirements/upper-constraints.txt" \
     -e git_branch={{ openstack_branch_slashed }} \
     -e ipa_upstream_release={{ openstack_branch }} \
     -e enable_inspector=true \
diff --git a/docker/kolla-toolbox/Dockerfile.j2 b/docker/kolla-toolbox/Dockerfile.j2
index 7a60674af..29c3ab640 100644
--- a/docker/kolla-toolbox/Dockerfile.j2
+++ b/docker/kolla-toolbox/Dockerfile.j2
@@ -78,9 +78,13 @@ RUN cd /usr/bin && \
     ln -s python3.12 python3
 {% endif %}
 
+{% block kolla_toolbox_pip_conf %}
+ENV UPPER_CONSTRAINTS_FILE=https://releases.openstack.org/constraints/upper/{{ openstack_release }}
+{% endblock %}
+
 {% block kolla_toolbox_upper_constraints %}
 RUN mkdir -p /requirements \
-    && curl -o /requirements/upper-constraints.txt ${UPPER_CONSTRAINTS_FILE:-https://releases.openstack.org/constraints/upper/{{ openstack_release }}} \
+    && curl -o /requirements/upper-constraints.txt $UPPER_CONSTRAINTS_FILE \
     && {{ macros.upper_constraints_remove("openstacksdk") }} \
     && python3 -m venv --system-site-packages {{ venv_path }} \
     && KOLLA_DISTRO_PYTHON_VERSION=$(/usr/bin/python3 -c "import sys; print('{}.{}'.format(sys.version_info.major, sys.version_info.minor))") \
diff --git a/releasenotes/notes/change-bifrost-upper-constraints-source-e93cb72e88823d56.yaml b/releasenotes/notes/change-bifrost-upper-constraints-source-e93cb72e88823d56.yaml
new file mode 100644
index 000000000..0b3b23354
--- /dev/null
+++ b/releasenotes/notes/change-bifrost-upper-constraints-source-e93cb72e88823d56.yaml
@@ -0,0 +1,14 @@
+---
+features:
+  - |
+    Updated the contents of Python package options in documentation to reflect
+    latest customisation methods.
+upgrade:
+  - |
+    The use of ``UPPER_CONSTRAINTS_FILE`` environment variable was moved from
+    ``base_pip_conf`` block in base Dockerfile to ``kolla_toolbox_pip_conf``
+    block in kolla-toolbox Dockerfile as the environment variable now only
+    affects Kolla-toolbox build.
+    Bifrost python upper-constraints now follows openstack-base's
+    python upper-constraints instead of ``UPPER_CONSTRAINTS_FILE`` environment
+    variable.