CI: make it work again

Python changes:
- limit pip to 20.x as this is last version working on both Python 2 and 3.
- cap 'setuptools' to be older than 58 to have use_2to3 functionality
  (fixes 'anyjson' installation)
- upgrade pip in 'elasticsearch-curator' image to get it built on CentOS 7

Unbuildable images:
- 'bifrost-base' under CentOS 8
  (Error: Cannot find a valid baseurl for repo: epel)
- 'monasca-grafana' under CentOS 7/8
- 'sensu-client' under CentOS 7
  (aws-eventstream requires Ruby version >= 2.3)

- dropped *-upgrade CI jobs as Stein is EOL

- pin 'qpid-proton-c' package for CentOS 7 to not install EPEL one
  (cinder-volume, openstack-base and tripleoclient)

- bump 'libvirt-python' in 'masakari-monitors' to 7.10.0
  (newer branches install package from distribution but CentOS 7 lacks
  it)

- pin 'unf_ext' gem in Sensu images as CentOS 7 lacks Ruby 2.2+
  (fixed sensu-server)

- install 'ca-certificates' for newer Let's Encrypt certificates support
  (so we can fetch upper-constraints)

- make Zuul fetch openstack/requirements project files
   (make 'openstack-base' pass) (like 2nd fix)

Co-Authored-by: Pierre Riteau <pierre@stackhpc.com>
Closes-Bug: #1943617
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/826464

Change-Id: I8ee32e53cda581493b2a4bb73ca72ab05014b9d4

2 files changed, 15 insertions, 2 deletions

diff --git a/tests/playbooks/pre.yml b/tests/playbooks/pre.yml
index 31944e992..afcf8f228 100644
--- a/tests/playbooks/pre.yml
+++ b/tests/playbooks/pre.yml
@@ -44,6 +44,12 @@
           - python-wheel
           - python-virtualenv
 
+    - name: Install CA certificates (to have up-to-date Let's Encrypt support)
+      become: true
+      package:
+        name:
+          - ca-certificates
+
     - name: Install virtualenv on Debian systems
       # NOTE(hrw): On RedHat systems it is part of python3-virtualenv
       package:
@@ -57,7 +63,7 @@
       # NOTE(mnasiadka): pip 8.x delivered with EPEL has problems installing
       # zipp and configparser
       become: true
-      command: "pip install --upgrade pip"
+      command: "pip install --upgrade pip==20.*"
 
     - name: Ensure tox is installed
       pip:
diff --git a/tests/playbooks/run.yml b/tests/playbooks/run.yml
index 615990172..cdb6c11da 100644
--- a/tests/playbooks/run.yml
+++ b/tests/playbooks/run.yml
@@ -32,7 +32,14 @@
         src: "{{ zuul.executor.work_root }}/{{ zuul.project.src_dir }}/tests/templates/template_overrides.j2"
         dest: /etc/kolla/template_overrides.j2
 
+    # NOTE(hrw) REQUESTS_CA_BUNDLE is to get Let's Encrypt certificates trusted
     - name: Run tox
-      command: "{{ ansible_user_dir }}/tox-venv/bin/tox -e {{ action }}-{{ base_distro }}{{ base_tag | default('') }}-{{ install_type }}"
+      shell: |
+        if [ 'RedHat' == '{{ ansible_os_family }}' ]; then
+            export REQUESTS_CA_BUNDLE="/etc/pki/tls/certs/ca-bundle.crt"
+        fi
+        {{ ansible_user_dir }}/tox-venv/bin/tox -e {{ action }}-{{ base_distro }}{{ base_tag | default('') }}-{{ install_type }}
       args:
         chdir: "{{ zuul.project.src_dir }}"
+      environment:
+        UPPER_CONSTRAINTS_FILE: "{{ requirements_src_dir }}/upper-constraints.txt"