diff options
174 files changed, 1989 insertions, 793 deletions
diff --git a/.yamllint b/.yamllint new file mode 100644 index 000000000..016ae71e8 --- /dev/null +++ b/.yamllint @@ -0,0 +1,27 @@ +extends: default + +rules: + comments: + # https://github.com/prettier/prettier/issues/6780 + min-spaces-from-content: 1 + # https://github.com/adrienverge/yamllint/issues/384 + comments-indentation: false + document-start: disable + # 160 chars was the default used by old E204 rule, but + # you can easily change it or disable in your .yamllint file. + line-length: + ignore: | + kolla/common/sources.yaml + max: 160 + # We are adding an extra space inside braces as that's how prettier does it + # and we are trying not to fight other linters. + braces: + min-spaces-inside: 0 # yamllint defaults to 0 + max-spaces-inside: 1 # yamllint defaults to 0 + # key-duplicates: + # forbid-duplicated-merge-keys: true # not enabled by default + octal-values: + forbid-implicit-octal: true # yamllint defaults to false + forbid-explicit-octal: true # yamllint defaults to false + # quoted-strings: + # quote-type: double diff --git a/doc/source/admin/image-building.rst b/doc/source/admin/image-building.rst index 3a40f5f6e..a3d9207d3 100644 --- a/doc/source/admin/image-building.rst +++ b/doc/source/admin/image-building.rst @@ -25,7 +25,7 @@ In general, images are built like this: kolla-build -By default, the above command would build all images based on a CentOS Stream +By default, the above command would build all images based on a Rocky Linux image. The operator can change the base distro with the ``-b`` option: @@ -36,7 +36,6 @@ The operator can change the base distro with the ``-b`` option: There are following distros (bases) available for building images: -- centos - debian - rocky - ubuntu @@ -704,8 +703,8 @@ additional configuration. For RPM-based distros they are enabled via ``dnf config-manager --enable``; for Debian-based distros they are a no-op as the sources are already provided by the base image. Examples include ``crb``, ``extras``, and the base repos ``baseos`` and ``appstream`` for -CentOS and Rocky Linux, as well as ``debian``, ``debian-security``, -``ubuntu``, ``ubuntu-security``. +Rocky Linux, as well as ``debian``, ``debian-security``, ``ubuntu``, +``ubuntu-security``. Repositories without ``distro: True`` are fully templated into a new repository file and must provide at least one of ``baseurl``, ``metalink``, @@ -783,6 +782,52 @@ Kolla validates this at build time and raises an error listing the missing overrides. For Rocky Linux this means overriding ``baseos`` or ``appstream`` requires also overriding ``crb``. +Build-time-only repositories +"""""""""""""""""""""""""""" + +Repositories that point to CI mirrors or other infrastructure-internal URLs +should not be baked into the final image — they may be unreachable at runtime +and would expose internal infrastructure details. Mark such repositories with +``build_only: true`` to have kolla automatically restore or remove the +repository file at the end of each image build. + +When ``build_only: true`` is set on a repository: + +* **Before** writing the mirror configuration, kolla backs up the existing + repository file (if any) to ``/tmp/kolla-repos-backup/``. +* The repository file is written normally so all package installations in that + image can use the mirror. +* **After** all package installations, kolla injects a cleanup step into every + rendered Dockerfile that restores the original file from the backup, or + removes the file entirely if no original existed. + +This means the repository files in the final image contain the upstream URLs +(or are absent, leaving the distro defaults in place) rather than mirror URLs. + +Example — Debian with a CI mirror marked build-time-only: + +.. code-block:: yaml + + debian: + debian: + build_only: true + component: "main" + gpg_key: "/usr/share/keyrings/debian-archive-keyring.gpg" + suite: "trixie trixie-updates trixie-backports" + url: "http://ci-mirror.internal/debian" + debian-security: + build_only: true + component: "main" + gpg_key: "/usr/share/keyrings/debian-archive-keyring.gpg" + suite: "trixie-security" + url: "http://ci-mirror.internal/debian-security" + +.. note:: + + ``build_only: true`` is only effective for non-distro repository entries + (those that write a ``.repo`` or ``.sources`` file). Repositories with + ``distro: True`` are a no-op in ``handle_repos()`` and require no cleanup. + Red Hat ^^^^^^^ diff --git a/doc/source/admin/kolla_api.rst b/doc/source/admin/kolla_api.rst index a34cd819f..b4266aea0 100644 --- a/doc/source/admin/kolla_api.rst +++ b/doc/source/admin/kolla_api.rst @@ -188,5 +188,5 @@ Variables available in the containers The following variables available in all images and can be evaluated in scripts: -* **KOLLA_BASE_DISTRO**: ``base_distro`` used to build the image (e.g. centos, +* **KOLLA_BASE_DISTRO**: ``base_distro`` used to build the image (e.g. rocky, ubuntu) diff --git a/doc/source/admin/template-override/ovs-dpdk.rst b/doc/source/admin/template-override/ovs-dpdk.rst index 73851a993..766caf9bc 100644 --- a/doc/source/admin/template-override/ovs-dpdk.rst +++ b/doc/source/admin/template-override/ovs-dpdk.rst @@ -1,7 +1,7 @@ OVS-DPDK Source build ===================== -CentOS currently does not provide packages for ovs with dpdk. +Rocky currently does not provide packages for ovs with dpdk. The Ubuntu packages do not support UIO based drivers. To use the uio_pci_generic driver on Ubuntu a source build is required. diff --git a/doc/source/contributor/adding-a-new-image.rst b/doc/source/contributor/adding-a-new-image.rst index 995bf13c7..072d3f294 100644 --- a/doc/source/contributor/adding-a-new-image.rst +++ b/doc/source/contributor/adding-a-new-image.rst @@ -75,7 +75,7 @@ cannot be built for some distribution/architecture/build-type combinations. "bifrost-base", # someone need to get upstream working first }, - 'centos': { + 'rocky': { "hacluster-pcs", # Missing crmsh package }, diff --git a/doc/source/contributor/versions.rst b/doc/source/contributor/versions.rst index af5d02f10..f553e18ed 100644 --- a/doc/source/contributor/versions.rst +++ b/doc/source/contributor/versions.rst @@ -9,9 +9,6 @@ information about those which come from 3rdparty sources. For each component used we list version used at branch release and provide information about package sources. -.. note:: - When table mentions 'CentOS' it means both CentOS Stream 10 and Rocky Linux 10. - ============== ================ ============================================= Name Version Package source information ============== ================ ============================================= @@ -20,13 +17,13 @@ information about package sources. Galera 26.4 (LTS) `MariaDB Community downloads`_ OpenSearch 3.x `OpenSearch install guide`_ ProxySQL 3.0.x `ProxySQL repository`_ - RabbitMQ 4.3.x - CentOS/Rocky: + RabbitMQ 4.3.x - Rocky: `Team RabbitMQ 'Cloudsmith' repo (RPM)`_ - Debian/Ubuntu: `Team RabbitMQ 'Cloudsmith' repo (Deb)`_ - Erlang 27.x - CentOS/Rocky aarch64: + Erlang 27.x - Rocky aarch64: `openstack-kolla COPR`_ - - CentOS/Rocky x86-64: + - Rocky x86-64: `Team RabbitMQ 'Cloudsmith' repo (RPM)`_ - Debian/Ubuntu: `Team RabbitMQ 'Modern Erlang' PPA`_ diff --git a/doc/source/support_matrix.rst b/doc/source/support_matrix.rst index c14c71954..5e3b06b76 100644 --- a/doc/source/support_matrix.rst +++ b/doc/source/support_matrix.rst @@ -17,7 +17,6 @@ The following base container images are supported: ================== =============================== ================ Distribution Default base Default base tag ================== =============================== ================ -Centos quay.io/centos/centos stream10 Debian Trixie debian trixie Rocky Linux quay.io/rockylinux/rockylinux 10 Ubuntu Noble ubuntu 24.04 diff --git a/kolla/common/config.py b/kolla/common/config.py index 1915f5c48..e1c0cb456 100644 --- a/kolla/common/config.py +++ b/kolla/common/config.py @@ -21,18 +21,16 @@ from kolla.common.users import USERS from kolla.version import version_info as version -BASE_OS_DISTRO = ['centos', 'debian', 'rocky', 'ubuntu'] +BASE_OS_DISTRO = ['debian', 'rocky', 'ubuntu'] BASE_ARCH = ['x86_64', 'aarch64'] DEBIAN_ARCH = ['amd64', 'arm64'] DEFAULT_BASE_TAGS = { - 'centos': {'name': 'quay.io/centos/centos', 'tag': 'stream10'}, 'debian': {'name': 'debian', 'tag': 'trixie'}, 'rocky': {'name': 'quay.io/rockylinux/rockylinux', 'tag': '10'}, 'ubuntu': {'name': 'ubuntu', 'tag': '24.04'}, } # NOTE(hrw): has to match PRETTY_NAME in /etc/os-release DISTRO_PRETTY_NAME = { - 'centos': 'CentOS Stream 10', 'debian': 'Debian GNU/Linux 13 (trixie)', 'rocky': 'Rocky Linux 10.* (Red Quartz)', 'ubuntu': 'Ubuntu 24.04.* LTS', diff --git a/kolla/common/sources.py b/kolla/common/sources.py index 59d2796f2..61dd10263 100644 --- a/kolla/common/sources.py +++ b/kolla/common/sources.py @@ -10,440 +10,11 @@ # See the License for the specific language governing permissions and # limitations under the License. -SOURCES = { - 'openstack-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/requirements/' - 'requirements-${openstack_branch}.tar.gz')}, - 'aodh-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/aodh/' - 'aodh-${openstack_branch}.tar.gz')}, - 'barbican-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/barbican/' - 'barbican-${openstack_branch}.tar.gz')}, - 'bifrost-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/bifrost/' - 'bifrost-${openstack_branch}.tar.gz')}, - 'blazar-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/blazar/' - 'blazar-${openstack_branch}.tar.gz')}, - 'ceilometer-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/ceilometer/' - 'ceilometer-${openstack_branch}.tar.gz')}, - 'cinder-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/cinder/' - 'cinder-${openstack_branch}.tar.gz')}, - 'cloudkitty-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/cloudkitty/' - 'cloudkitty-${openstack_branch}.tar.gz')}, - 'cyborg-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/cyborg/' - 'cyborg-${openstack_branch}.tar.gz')}, - 'designate-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/designate/' - 'designate-${openstack_branch}.tar.gz')}, - 'etcd': { - # NOTE(wszumski): Upgrade one minor version at a time: - # https://etcd.io/docs/v3.6/upgrades/upgrade_3_6/ - # NOTE(mnasiadka): lts_branch pins check_sources.py to the 3.6.x - # release series instead of jumping to the latest major.minor. - 'lts_branch': '3.6', - 'version': '3.6.5', - 'type': 'url', - 'sha256': { - 'amd64': '66bad39ed920f6fc15fd74adcb8bfd38ba9a6412f8c7852d09eb11670e88cac3', # noqa: E501 - 'arm64': '7010161787077b07de29b15b76825ceacbbcedcb77fe2e6832f509be102cab6b'}, # noqa: E501 - 'location': ('https://github.com/etcd-io/etcd/' - 'releases/download/v${version}' - '/etcd-v${version}-linux-${debian_arch}.tar.gz')}, - 'glance-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/glance/' - 'glance-${openstack_branch}.tar.gz')}, - 'gnocchi-base': { - 'type': 'git', - 'reference': '4.7.0', - 'location': ('https://github.com/gnocchixyz/' - 'gnocchi.git')}, - 'heat-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/heat/' - 'heat-${openstack_branch}.tar.gz')}, - 'horizon': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/horizon/' - 'horizon-${openstack_branch}.tar.gz')}, - 'horizon-plugin-blazar-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/blazar-dashboard/' - 'blazar-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-cloudkitty-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/cloudkitty-dashboard/' - 'cloudkitty-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-designate-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/designate-dashboard/' - 'designate-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-fwaas-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/neutron-fwaas-dashboard/' - 'neutron-fwaas-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-heat-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/heat-dashboard/' - 'heat-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-ironic-ui': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/ironic-ui/' - 'ironic-ui-${openstack_branch}.tar.gz')}, - 'horizon-plugin-magnum-ui': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/magnum-ui/' - 'magnum-ui-${openstack_branch}.tar.gz')}, - 'horizon-plugin-manila-ui': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/manila-ui/' - 'manila-ui-${openstack_branch}.tar.gz')}, - 'horizon-plugin-masakari-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/masakari-dashboard/' - 'masakari-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-mistral-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/mistral-dashboard/' - 'mistral-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-neutron-vpnaas-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/neutron-vpnaas-dashboard/' - 'neutron-vpnaas-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-octavia-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/octavia-dashboard/' - 'octavia-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-tacker-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/tacker-horizon/' - 'tacker-horizon-${openstack_branch}.tar.gz')}, - 'horizon-plugin-trove-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/trove-dashboard/' - 'trove-dashboard-${openstack_branch}.tar.gz')}, - 'horizon-plugin-watcher-dashboard': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/watcher-dashboard/' - 'watcher-dashboard-${openstack_branch}.tar.gz')}, - 'ironic-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/ironic/' - 'ironic-${openstack_branch}.tar.gz')}, - 'ironic-conductor-plugin-prometheus-exporter': { - 'type': 'url', - 'location': ( - '$tarballs_base/openstack/ironic-prometheus-exporter/' - 'ironic-prometheus-exporter-${openstack_branch}.tar.gz')}, - 'keystone-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/keystone/' - 'keystone-${openstack_branch}.tar.gz')}, - 'kerbside-base': { - 'type': 'git', - 'reference': 'develop', - 'location': ('https://github.com/shakenfist/kerbside')}, - 'kolla-toolbox': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/requirements/' - 'requirements-${openstack_branch}.tar.gz')}, - 'letsencrypt-lego': { - 'version': 'v4.27.0', - 'type': 'url', - 'sha256': { - 'amd64': '898b58bbbca4282d706b4f204593cb94fc2ed13232777236c06dc20259bbcd02', # noqa: E501 - 'arm64': '2973b412d37e5d652a91bda1a6bf7642491e316bc0f855a614e2c996249014dc'}, # noqa: E501 - 'location': ('https://github.com/go-acme/lego/' - 'releases/download/${version}/' - 'lego_${version}_linux_${debian_arch}.tar.gz')}, - 'magnum-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/magnum/' - 'magnum-${openstack_branch}.tar.gz')}, - 'magnum-conductor-plugin-helm': { - 'version': 'v3.19.0', - 'type': 'url', - # NOTE(mnasiadka): location points to get.helm.sh, but - # check_sources needs github releases access - 'github': 'helm/helm', - 'sha256': { - 'amd64': 'a7f81ce08007091b86d8bd696eb4d86b8d0f2e1b9f6c714be62f82f96a594496', # noqa: E501 - 'arm64': '440cf7add0aee27ebc93fada965523c1dc2e0ab340d4348da2215737fc0d76ad'}, # noqa: E501 - 'location': ('https://get.helm.sh/helm' - '-${version}-linux-${debian_arch}.tar.gz')}, - 'manila-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/manila/' - 'manila-${openstack_branch}.tar.gz')}, - 'mariadb-server-plugin-mariadb-docker': { - # NOTE(seunghun1ee): This repo is needed to get MariaDB healthcheck.sh - 'type': 'git', - 'reference': 'master', - 'location': ('https://github.com/MariaDB/mariadb-docker')}, - 'masakari-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/masakari/' - 'masakari-${openstack_branch}.tar.gz')}, - 'masakari-monitors': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/masakari-monitors/' - 'masakari-monitors-${openstack_branch}.tar.gz')}, - 'mistral-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/mistral/' - 'mistral-${openstack_branch}.tar.gz')}, - 'mistral-base-plugin-tacker': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/tacker/' - 'tacker-${openstack_branch}.tar.gz')}, - 'neutron-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/neutron/' - 'neutron-${openstack_branch}.tar.gz')}, - 'neutron-base-plugin-neutron-fwaas': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/neutron-fwaas/' - 'neutron-fwaas-${openstack_branch}.tar.gz')}, - 'neutron-base-plugin-networking-baremetal': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/networking-baremetal/' - 'networking-baremetal-${openstack_branch}.tar.gz')}, - 'neutron-base-plugin-networking-generic-switch': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/networking-generic-switch/' - 'networking-generic-switch-${openstack_branch}.tar.gz')}, - 'neutron-base-plugin-networking-sfc': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/networking-sfc/' - 'networking-sfc-${openstack_branch}.tar.gz')}, - 'neutron-base-plugin-tap-as-a-service': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/tap-as-a-service/' - 'tap-as-a-service-${openstack_branch}.tar.gz')}, - 'neutron-base-plugin-vpnaas-agent': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/neutron-vpnaas/' - 'neutron-vpnaas-${openstack_branch}.tar.gz')}, - 'neutron-bgp-dragent': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/neutron-dynamic-routing/' - 'neutron-dynamic-routing-${openstack_branch}.tar.gz')}, - 'neutron-server-plugin-neutron-dynamic-routing': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/neutron-dynamic-routing/' - 'neutron-dynamic-routing-${openstack_branch}.tar.gz')}, - 'neutron-vpnaas-agent': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/neutron-vpnaas/' - 'neutron-vpnaas-${openstack_branch}.tar.gz')}, - 'nova-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/nova/' - 'nova-${openstack_branch}.tar.gz')}, - 'nova-base-plugin-blazar': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/blazar-nova/' - 'blazar-nova-${openstack_branch}.tar.gz')}, - 'octavia-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/octavia/' - 'octavia-${openstack_branch}.tar.gz')}, - 'octavia-api-plugin-ovn-octavia-provider': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/ovn-octavia-provider/' - 'ovn-octavia-provider-${openstack_branch}.tar.gz')}, - 'octavia-driver-agent-plugin-ovn-octavia-provider': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/ovn-octavia-provider/' - 'ovn-octavia-provider-${openstack_branch}.tar.gz')}, - 'placement-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/placement/' - 'placement-${openstack_branch}.tar.gz')}, - 'prometheus-alertmanager': { - 'version': '0.31.1', - 'type': 'url', - 'sha256': { - 'amd64': '35191cbd9d4f8162458b78dd7e93990cccd246044d9a6f788adb1c66ac3ea07b', # noqa: E501 - 'arm64': '266dda88b64318c27847ef9af4ff450fc178c827550a8039420c5ca8657a4a8b'}, # noqa: E501 - 'location': ('https://github.com/' - 'prometheus/alertmanager/' - 'releases/download/v${version}/' - 'alertmanager' - '-${version}.linux-${debian_arch}.tar.gz')}, - 'prometheus-blackbox-exporter': { - 'version': '0.28.0', - 'type': 'url', - 'sha256': { - 'amd64': 'caf5d242fb1cf6d5cb678f3f799f22703d4fafea26b03dcbbd7e1f1825e06329', # noqa: E501 - 'arm64': '63312be0983d85e5109710a7dc93df3051157ae581853fa3655d171cc1b2806e'}, # noqa: E501 - 'location': ('https://github.com/' - 'prometheus/blackbox_exporter/' - 'releases/download/v${version}/' - 'blackbox_exporter' - '-${version}.linux-${debian_arch}.tar.gz')}, - 'prometheus-cadvisor': { - 'version': '0.56.2', - 'type': 'url', - 'sha256': { - 'amd64': 'ad92930f16a2f9da15190675e09eeaceb8fd38637d07a686bb0dd68695f692af', # noqa: E501 - 'arm64': 'b7a707379496fd7a7b5d2768c5c494427112f534ba5069f889af28ffe6ad11bb'}, # noqa: E501 - 'location': ('https://github.com/' - 'google/cadvisor/' - 'releases/download/v${version}/' - 'cadvisor' - '-v${version}-linux-${debian_arch}')}, - 'prometheus-elasticsearch-exporter': { - 'version': '1.10.0', - 'type': 'url', - 'sha256': { - 'amd64': '1dcf288082a25b2741e98da6c9fc012b6c821696a26c6ac57c20042f24714a74', # noqa: E501 - 'arm64': '0ff4753a975eb5611c03123d565e8aaa84e4c05f698ce0a2d6c0f437a14bfe34'}, # noqa: E501 - 'location': ('https://github.com/' - 'prometheus-community/elasticsearch_exporter/' - 'releases/download/v${version}/' - 'elasticsearch_exporter' - '-${version}.linux-${debian_arch}.tar.gz')}, - 'prometheus-libvirt-exporter': { - 'version': '2.3.1', - 'type': 'url', - 'sha256': { - 'amd64': '2e1657d10291059f420a467cbe3363c7353bbce9d5445f0d8d302bd96caaf0a8', # noqa: E501 - 'arm64': '676dc76766ce4a2eaa8324a3aca8cfe89711536337d3b5b862ddf141fb57a224'}, # noqa: E501 - 'location': ('https://github.com/' - 'inovex/prometheus-libvirt-exporter/' - 'releases/download/v${version}/' - 'prometheus-libvirt-exporter' - '-${version}.linux-${debian_arch}.tar.gz')}, - 'prometheus-memcached-exporter': { - 'version': '0.15.5', - 'type': 'url', - 'sha256': { - 'amd64': '5b82e6579b77fae00683d10d5923329173925a5abd95a799f5489a66136b884c', # noqa: E501 - 'arm64': '609e6d7484f22fd9e405c0022a9e2d7f86eedef1cfa55772231528271082aeae'}, # noqa: E501 - 'location': ('https://github.com/' - 'prometheus/memcached_exporter/' - 'releases/download/v${version}/' - 'memcached_exporter' - '-${version}.linux-${debian_arch}.tar.gz')}, - 'prometheus-mtail': { - 'version': '3.4.0', - 'type': 'url', - 'sha256': { - 'amd64': 'd3d0d34909d03ffba11baccd87989c1825fc5c68b1f59307fe188a6dd8e0da5a', # noqa: E501 - 'arm64': '2b166f4d4165b508b9c9b96de6364bc35ff019545810ad07d919ee5ea3d064c7'}, # noqa: E501 - 'location': ('https://github.com/' - 'jaqx0r/mtail/' - 'releases/download/v${version}/' - 'mtail' - '_${version}_linux_${debian_arch}.tar.gz')}, - 'prometheus-mysqld-exporter': { - 'version': '0.18.0', - 'type': 'url', - 'sha256': { - 'amd64': '46e8f45654352bdd42d162b2b4a68f00055d45acc168f9c068235b9e3acc39c1', # noqa: E501 - 'arm64': 'abdb452600ca086b68244aadf8045fbf0b6a48dcb76eed5576995806c176f6ce'}, # noqa: E501 - 'location': ('https://github.com/' - 'prometheus/mysqld_exporter/' - 'releases/download/v${version}/' - 'mysqld_exporter' - '-${version}.linux-${debian_arch}.tar.gz')}, - 'prometheus-node-exporter': { - 'version': '1.10.2', - 'type': 'url', - 'sha256': { - 'amd64': 'c46e5b6f53948477ff3a19d97c58307394a29fe64a01905646f026ddc32cb65b', # noqa: E501 - 'arm64': 'de69ec8341c8068b7c8e4cfe3eb85065d24d984a3b33007f575d307d13eb89a6'}, # noqa: E501 - 'location': ('https://github.com/' - 'prometheus/node_exporter/' - 'releases/download/v${version}/' - 'node_exporter' - '-${version}.linux-${debian_arch}.tar.gz')}, - 'prometheus-openstack-exporter': { - 'version': '1.7.0', - 'type': 'url', - 'sha256': { - 'amd64': 'dfaa0d3dcff22e882d3f61c56bb9ac6f70790df9d67361464159bbb4c7223192', # noqa: E501 - 'arm64': 'd6e0b23fe755732a93796255e3a2be8ec5a699b0a64c21afd377c60ccf60cd55'}, # noqa: E501 - 'location': ('https://github.com/' - 'openstack-exporter/openstack-exporter/' - 'releases/download/v${version}/' - 'openstack-exporter' - '_${version}_linux_${debian_arch}.tar.gz')}, - 'prometheus-openstack-network-exporter': { - 'version': '0.2.0', - 'type': 'url', - 'sha256': { - 'amd64': 'c93e6c62a0b861af50292f51b1443114e2178d89450b37cb0462723eaa3b2338', # noqa: E501 - 'arm64': '478de6e005188d0e2545e34ca2c5b93a33a32dc7d1f1b03dc3210c4ef7e44305'}, # noqa: E501 - 'location': ('https://github.com/' - 'openstack-k8s-operators/openstack-network-exporter/' - 'releases/download/v${version}/' - 'openstack-network-exporter' - '-linux-${debian_arch}')}, - 'prometheus-server': { - # NOTE(mnasiadka): lts_branch pins check_sources.py to the 3.5.x - # release series instead of jumping to the latest major.minor. - 'lts_branch': '3.5', - 'version': '3.5.4', - 'type': 'url', - 'sha256': { - 'amd64': 'be64e1cf657e6e7d132aca022f6135abedd660db053952e3f69e8affa3b9cc9e', # noqa: E501 - 'arm64': '97634b9b6ed9ef18689e3b3572f1b277714650657516494fd164dcc53c1d3f48'}, # noqa: E501 - 'location': ('https://github.com/' - 'prometheus/prometheus/' - 'releases/download/v${version}/' - 'prometheus' - '-${version}.linux-${debian_arch}.tar.gz')}, - 'prometheus-valkey-exporter': { - 'version': '1.82.0', - 'type': 'url', - 'sha256': { - 'amd64': '2d98dd0888f46e206bf3ad1b7c1784934a3a39cd8774cf47615f4c15594547cd', # noqa: E501 - 'arm64': '13aaa6acbb145848cb97ed3bab22747e094b3f0ed22df83c8ace593e3c527c95'}, # noqa: E501 - 'location': ('https://github.com/' - 'oliver006/redis_exporter/' - 'releases/download/v${version}/' - 'redis_exporter' - '-v${version}.linux-${debian_arch}.tar.gz')}, - 'skyline-apiserver': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/skyline-apiserver/' - 'skyline-apiserver-${openstack_branch}.tar.gz')}, - 'skyline-console': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/skyline-console/' - 'skyline-console-${openstack_branch}.tar.gz')}, - 'tacker-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/tacker/' - 'tacker-${openstack_branch}.tar.gz')}, - 'tacker-base-plugin-networking-sfc': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/networking-sfc/' - 'networking-sfc-${openstack_branch}.tar.gz')}, - 'trove-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/trove/' - 'trove-${openstack_branch}.tar.gz')}, - 'watcher-base': { - 'type': 'url', - 'location': ('$tarballs_base/openstack/watcher/' - 'watcher-${openstack_branch}.tar.gz')} -} +import os + +import yaml + +SOURCES_YAML = os.path.dirname(os.path.realpath(__file__)) + '/sources.yaml' + +with open(SOURCES_YAML, 'r') as f: + SOURCES = yaml.safe_load(f) diff --git a/kolla/common/sources.yaml b/kolla/common/sources.yaml new file mode 100644 index 000000000..8fd2ecf84 --- /dev/null +++ b/kolla/common/sources.yaml @@ -0,0 +1,318 @@ +--- +openstack-base: + type: "url" + location: "$tarballs_base/openstack/requirements/requirements-${openstack_branch}.tar.gz" +aodh-base: + type: "url" + location: "$tarballs_base/openstack/aodh/aodh-${openstack_branch}.tar.gz" +barbican-base: + type: "url" + location: "$tarballs_base/openstack/barbican/barbican-${openstack_branch}.tar.gz" +bifrost-base: + type: "url" + location: "$tarballs_base/openstack/bifrost/bifrost-${openstack_branch}.tar.gz" +blazar-base: + type: "url" + location: "$tarballs_base/openstack/blazar/blazar-${openstack_branch}.tar.gz" +ceilometer-base: + type: "url" + location: "$tarballs_base/openstack/ceilometer/ceilometer-${openstack_branch}.tar.gz" +cinder-base: + type: "url" + location: "$tarballs_base/openstack/cinder/cinder-${openstack_branch}.tar.gz" +cloudkitty-base: + type: "url" + location: "$tarballs_base/openstack/cloudkitty/cloudkitty-${openstack_branch}.tar.gz" +cyborg-base: + type: "url" + location: "$tarballs_base/openstack/cyborg/cyborg-${openstack_branch}.tar.gz" +designate-base: + type: "url" + location: "$tarballs_base/openstack/designate/designate-${openstack_branch}.tar.gz" +etcd: + # NOTE(wszumski): Upgrade one minor version at a time: + # https://etcd.io/docs/v3.6/upgrades/upgrade_3_6/ + # NOTE(mnasiadka): lts_branch pins check_sources.py to the 3.6.x + # release series instead of jumping to the latest major.minor. + lts_branch: "3.6" + version: "3.6.14" + type: "url" + sha256: + amd64: "ffe840ff9295808e88cce2794a18a5ac87f12a5203c8314d0bf6aa119b41bac5" + arm64: "fa1b80565ee6fc2df1ae6f57a508b221f3ecc7d591c317c581cd8281b0842b3e" + location: "https://github.com/etcd-io/etcd/releases/download/v${version}/etcd-v${version}-linux-${debian_arch}.tar.gz" +glance-base: + type: "url" + location: "$tarballs_base/openstack/glance/glance-${openstack_branch}.tar.gz" +gnocchi-base: + type: "git" + reference: "4.7.0" + location: "https://github.com/gnocchixyz/gnocchi.git" +heat-base: + type: "url" + location: "$tarballs_base/openstack/heat/heat-${openstack_branch}.tar.gz" +horizon: + type: "url" + location: "$tarballs_base/openstack/horizon/horizon-${openstack_branch}.tar.gz" +horizon-plugin-blazar-dashboard: + type: "url" + location: "$tarballs_base/openstack/blazar-dashboard/blazar-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-cloudkitty-dashboard: + type: "url" + location: "$tarballs_base/openstack/cloudkitty-dashboard/cloudkitty-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-designate-dashboard: + type: "url" + location: "$tarballs_base/openstack/designate-dashboard/designate-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-fwaas-dashboard: + type: "url" + location: "$tarballs_base/openstack/neutron-fwaas-dashboard/neutron-fwaas-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-heat-dashboard: + type: "url" + location: "$tarballs_base/openstack/heat-dashboard/heat-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-ironic-ui: + type: "url" + location: "$tarballs_base/openstack/ironic-ui/ironic-ui-${openstack_branch}.tar.gz" +horizon-plugin-magnum-ui: + type: "url" + location: "$tarballs_base/openstack/magnum-ui/magnum-ui-${openstack_branch}.tar.gz" +horizon-plugin-manila-ui: + type: "url" + location: "$tarballs_base/openstack/manila-ui/manila-ui-${openstack_branch}.tar.gz" +horizon-plugin-masakari-dashboard: + type: "url" + location: "$tarballs_base/openstack/masakari-dashboard/masakari-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-mistral-dashboard: + type: "url" + location: "$tarballs_base/openstack/mistral-dashboard/mistral-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-neutron-vpnaas-dashboard: + type: "url" + location: "$tarballs_base/openstack/neutron-vpnaas-dashboard/neutron-vpnaas-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-octavia-dashboard: + type: "url" + location: "$tarballs_base/openstack/octavia-dashboard/octavia-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-tacker-dashboard: + type: "url" + location: "$tarballs_base/openstack/tacker-horizon/tacker-horizon-${openstack_branch}.tar.gz" +horizon-plugin-trove-dashboard: + type: "url" + location: "$tarballs_base/openstack/trove-dashboard/trove-dashboard-${openstack_branch}.tar.gz" +horizon-plugin-watcher-dashboard: + type: "url" + location: "$tarballs_base/openstack/watcher-dashboard/watcher-dashboard-${openstack_branch}.tar.gz" +ironic-base: + type: "url" + location: "$tarballs_base/openstack/ironic/ironic-${openstack_branch}.tar.gz" +ironic-conductor-plugin-prometheus-exporter: + type: "url" + location: "$tarballs_base/openstack/ironic-prometheus-exporter/ironic-prometheus-exporter-${openstack_branch}.tar.gz" +keystone-base: + type: "url" + location: "$tarballs_base/openstack/keystone/keystone-${openstack_branch}.tar.gz" +kerbside-base: + type: "git" + reference: "develop" + location: "https://github.com/shakenfist/kerbside" +kolla-toolbox: + type: "url" + location: "$tarballs_base/openstack/requirements/requirements-${openstack_branch}.tar.gz" +letsencrypt-lego: + lts_branch: "4" + version: "v4.35.2" + type: "url" + sha256: + amd64: "ee5be4bf457de8e3efa86a51651c75c87f0ee0e4e9f3ae14f6034d68365770f3" + arm64: "e1f153179098d27ce044aaaa168c0e323d50ae71b0f1a147aa8ae49ac6b14d89" + location: "https://github.com/go-acme/lego/releases/download/${version}/lego_${version}_linux_${debian_arch}.tar.gz" +magnum-base: + type: "url" + location: "$tarballs_base/openstack/magnum/magnum-${openstack_branch}.tar.gz" +magnum-conductor-plugin-helm: + version: "v4.3.0" + type: "url" + # NOTE(mnasiadka): location points to get.helm.sh, but + # check_sources needs github releases access + github: "helm/helm" + sha256: + amd64: "86584a54def73570558f66f5111cc53dfed56689637ae32c1201205d494f54fb" + arm64: "31c5794dd55c66a51e6b7d2e2ac7a114ae8b1de41ff1d9ba51748ac973b06a08" + location: "https://get.helm.sh/helm-${version}-linux-${debian_arch}.tar.gz" +manila-base: + type: "url" + location: "$tarballs_base/openstack/manila/manila-${openstack_branch}.tar.gz" +mariadb-server-plugin-mariadb-docker: + # NOTE(seunghun1ee): This repo is needed to get MariaDB healthcheck.sh + type: "git" + reference: "master" + location: "https://github.com/MariaDB/mariadb-docker" +masakari-base: + type: "url" + location: "$tarballs_base/openstack/masakari/masakari-${openstack_branch}.tar.gz" +masakari-monitors: + type: "url" + location: "$tarballs_base/openstack/masakari-monitors/masakari-monitors-${openstack_branch}.tar.gz" +mistral-base: + type: "url" + location: "$tarballs_base/openstack/mistral/mistral-${openstack_branch}.tar.gz" +mistral-base-plugin-tacker: + type: "url" + location: "$tarballs_base/openstack/tacker/tacker-${openstack_branch}.tar.gz" +neutron-base: + type: "url" + location: "$tarballs_base/openstack/neutron/neutron-${openstack_branch}.tar.gz" +neutron-base-plugin-neutron-fwaas: + type: "url" + location: "$tarballs_base/openstack/neutron-fwaas/neutron-fwaas-${openstack_branch}.tar.gz" +neutron-base-plugin-networking-baremetal: + type: "url" + location: "$tarballs_base/openstack/networking-baremetal/networking-baremetal-${openstack_branch}.tar.gz" +neutron-base-plugin-networking-generic-switch: + type: "url" + location: "$tarballs_base/openstack/networking-generic-switch/networking-generic-switch-${openstack_branch}.tar.gz" +neutron-base-plugin-networking-sfc: + type: "url" + location: "$tarballs_base/openstack/networking-sfc/networking-sfc-${openstack_branch}.tar.gz" +neutron-base-plugin-tap-as-a-service: + type: "url" + location: "$tarballs_base/openstack/tap-as-a-service/tap-as-a-service-${openstack_branch}.tar.gz" +neutron-base-plugin-vpnaas-agent: + type: "url" + location: "$tarballs_base/openstack/neutron-vpnaas/neutron-vpnaas-${openstack_branch}.tar.gz" +neutron-bgp-dragent: + type: "url" + location: "$tarballs_base/openstack/neutron-dynamic-routing/neutron-dynamic-routing-${openstack_branch}.tar.gz" +neutron-server-plugin-neutron-dynamic-routing: + type: "url" + location: "$tarballs_base/openstack/neutron-dynamic-routing/neutron-dynamic-routing-${openstack_branch}.tar.gz" +neutron-vpnaas-agent: + type: "url" + location: "$tarballs_base/openstack/neutron-vpnaas/neutron-vpnaas-${openstack_branch}.tar.gz" +nova-base: + type: "url" + location: "$tarballs_base/openstack/nova/nova-${openstack_branch}.tar.gz" +nova-base-plugin-blazar: + type: "url" + location: "$tarballs_base/openstack/blazar-nova/blazar-nova-${openstack_branch}.tar.gz" +octavia-base: + type: "url" + location: "$tarballs_base/openstack/octavia/octavia-${openstack_branch}.tar.gz" +octavia-api-plugin-ovn-octavia-provider: + type: "url" + location: "$tarballs_base/openstack/ovn-octavia-provider/ovn-octavia-provider-${openstack_branch}.tar.gz" +octavia-driver-agent-plugin-ovn-octavia-provider: + type: "url" + location: "$tarballs_base/openstack/ovn-octavia-provider/ovn-octavia-provider-${openstack_branch}.tar.gz" +placement-base: + type: "url" + location: "$tarballs_base/openstack/placement/placement-${openstack_branch}.tar.gz" +prometheus-alertmanager: + version: "0.34.0" + type: "url" + sha256: + amd64: "19c75a11d8c03dc4ade7abdbddfb3a8f28c9e7b000d0849cda0cd71dffd74a03" + arm64: "a96ef16598ddc58e84d28167b3352b30e3205698d76c56ef467e4504d7664da4" + location: "https://github.com/prometheus/alertmanager/releases/download/v${version}/alertmanager-${version}.linux-${debian_arch}.tar.gz" +prometheus-blackbox-exporter: + version: "0.28.0" + type: "url" + sha256: + amd64: "caf5d242fb1cf6d5cb678f3f799f22703d4fafea26b03dcbbd7e1f1825e06329" + arm64: "63312be0983d85e5109710a7dc93df3051157ae581853fa3655d171cc1b2806e" + location: "https://github.com/prometheus/blackbox_exporter/releases/download/v${version}/blackbox_exporter-${version}.linux-${debian_arch}.tar.gz" +prometheus-cadvisor: + version: "0.60.5" + type: "url" + sha256: + amd64: "6f258bed28113218c88729634c47ead8d928b81b1504b75fd911b98d29003a87" + arm64: "6f93087c114bc64a7b8d5c4148190b4d9b95511b9bf023246ce4c76dc448a48a" + location: "https://github.com/google/cadvisor/releases/download/v${version}/cadvisor-v${version}-linux-${debian_arch}" +prometheus-elasticsearch-exporter: + version: "1.11.0" + type: "url" + sha256: + amd64: "2bebc50d6980578ad37a376000bc039485ceea129ae229f78baed3379db4f61c" + arm64: "7020d10e29896d329cecc7bfe2e90dcc376d68fc0c75ac27014af3747870ce1f" + location: "https://github.com/prometheus-community/elasticsearch_exporter/releases/download/v${version}/elasticsearch_exporter-${version}.linux-${debian_arch}.tar.gz" +prometheus-libvirt-exporter: + version: "2.5.0" + type: "url" + sha256: + amd64: "07a49f3d0424259eba88cbc41310017fbbbd6d8ce0b2a0e4dfad3d973b55f558" + arm64: "4d13a4a930adabc9c2be8c83c7a8bb2b3e81d8743e9b645590f3d28311e1e8cb" + location: "https://github.com/inovex/prometheus-libvirt-exporter/releases/download/v${version}/prometheus-libvirt-exporter-${version}.linux-${debian_arch}.tar.gz" +prometheus-memcached-exporter: + version: "0.17.0" + type: "url" + sha256: + amd64: "40e85262e4340a56fa7cc96722cd596d25ed76ab501459b0b42f1f8e0562d0d1" + arm64: "d1abf0ef1196ff5de5cf3462eb962b4a3d6733327951ae6d81c062902b5f2a60" + location: "https://github.com/prometheus/memcached_exporter/releases/download/v${version}/memcached_exporter-${version}.linux-${debian_arch}.tar.gz" +prometheus-mtail: + version: "3.4.12" + type: "url" + sha256: + amd64: "c5f1cca4891589ffa52a7c3a32e4802c1d4959e31626df953f3d388393720916" + arm64: "acfde7e7cdbf8ecb1865d0cc9bc9dfe7dd657822377bfd5f81405c93b5fedaab" + location: "https://github.com/jaqx0r/mtail/releases/download/v${version}/mtail_${version}_linux_${debian_arch}.tar.gz" +prometheus-mysqld-exporter: + version: "0.20.0" + type: "url" + sha256: + amd64: "5773496e9962ca3817b3599fe4d74f218c95c1295eb2a742462df8e035fe51bd" + arm64: "837804c0a59cbe1f3ab46670dec1b4c37940c8d48062374691b27f1f37abf8dd" + location: "https://github.com/prometheus/mysqld_exporter/releases/download/v${version}/mysqld_exporter-${version}.linux-${debian_arch}.tar.gz" +prometheus-node-exporter: + version: "1.12.1" + type: "url" + sha256: + amd64: "b51d8a76aa2a9156a55d501aca6276fae09e262259a5e4e831d2c2222f084e63" + arm64: "ad35b605f9954b9f1ffddf5ba054bdc5a98d790b9eae5291e1eeb83f1ecbd0e7" + location: "https://github.com/prometheus/node_exporter/releases/download/v${version}/node_exporter-${version}.linux-${debian_arch}.tar.gz" +prometheus-openstack-exporter: + version: "1.7.0" + type: "url" + sha256: + amd64: "dfaa0d3dcff22e882d3f61c56bb9ac6f70790df9d67361464159bbb4c7223192" + arm64: "d6e0b23fe755732a93796255e3a2be8ec5a699b0a64c21afd377c60ccf60cd55" + location: "https://github.com/openstack-exporter/openstack-exporter/releases/download/v${version}/openstack-exporter_${version}_linux_${debian_arch}.tar.gz" +prometheus-openstack-network-exporter: + version: "0.3.0" + type: "url" + sha256: + amd64: "bca401a6854bdc350578ff9e3477b4c54527c3f677d7549191e2523f1d03605d" + arm64: "06640f01fd088068f2552e702b624d4b9f2441d532409f8be7d94206fc18b9c0" + location: "https://github.com/openstack-k8s-operators/openstack-network-exporter/releases/download/v${version}/openstack-network-exporter-linux-${debian_arch}" +prometheus-server: + # NOTE(mnasiadka): lts_branch pins check_sources.py to the 3.13.x + # release series instead of jumping to the latest major.minor. + lts_branch: "3.13" + version: "3.13.3" + type: "url" + sha256: + amd64: "b349c732d8a853e657d0e7ae1bbad4d11b586615fb65fdc59d896b9f869c001e" + arm64: "aa9be9c36608e7ab7cc089726195d0e4fa74f87b330538df9d30d3da1f2b0e18" + location: "https://github.com/prometheus/prometheus/releases/download/v${version}/prometheus-${version}.linux-${debian_arch}.tar.gz" +prometheus-valkey-exporter: + version: "1.91.1" + type: "url" + sha256: + amd64: "4552451000b6345011bfc50d77fd3be625aa0be47ac07e98a40384038ecffaab" + arm64: "7ee124bd17733cce535b87da17e79f0346d3edb4a20759cf3bfca8b4e0d6886f" + location: "https://github.com/oliver006/redis_exporter/releases/download/v${version}/redis_exporter-v${version}.linux-${debian_arch}.tar.gz" +skyline-apiserver: + type: "url" + location: "$tarballs_base/openstack/skyline-apiserver/skyline-apiserver-${openstack_branch}.tar.gz" +skyline-console: + type: "url" + location: "$tarballs_base/openstack/skyline-console/skyline-console-${openstack_branch}.tar.gz" +tacker-base: + type: "url" + location: "$tarballs_base/openstack/tacker/tacker-${openstack_branch}.tar.gz" +tacker-base-plugin-networking-sfc: + type: "url" + location: "$tarballs_base/openstack/networking-sfc/networking-sfc-${openstack_branch}.tar.gz" +trove-base: + type: "url" + location: "$tarballs_base/openstack/trove/trove-${openstack_branch}.tar.gz" +watcher-base: + type: "url" + location: "$tarballs_base/openstack/watcher/watcher-${openstack_branch}.tar.gz" diff --git a/kolla/docker/aodh/aodh-base/Dockerfile.j2 b/kolla/docker/aodh/aodh-base/Dockerfile.j2 index 519537468..b2d863bb9 100644 --- a/kolla/docker/aodh/aodh-base/Dockerfile.j2 +++ b/kolla/docker/aodh/aodh-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='aodh') }} {% set aodh_base_packages = [ @@ -36,3 +38,4 @@ RUN ln -s aodh-base-source/* aodh \ {{ macros.kolla_patch_sources() }} {% block aodh_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/barbican/barbican-base/Dockerfile.j2 b/kolla/docker/barbican/barbican-base/Dockerfile.j2 index 89c44ef9a..e5edbb8a9 100644 --- a/kolla/docker/barbican/barbican-base/Dockerfile.j2 +++ b/kolla/docker/barbican/barbican-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='barbican', groups='nfast,hsmusers') }} {{ macros.install_packages(barbican_base_packages | customizable("packages")) }} @@ -34,3 +36,4 @@ RUN ln -s barbican-base-source/* barbican \ {{ macros.kolla_patch_sources() }} {% block barbican_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/base/Dockerfile.j2 b/kolla/docker/base/Dockerfile.j2 index 746f0e731..c1f07ade6 100644 --- a/kolla/docker/base/Dockerfile.j2 +++ b/kolla/docker/base/Dockerfile.j2 @@ -58,12 +58,12 @@ COPY dnf.conf /etc/dnf/dnf.conf {% set base_yum_repo_keys = [ ] %} -{% block base_centos_repo_overrides_post_copy %}{% endblock %} +{% block base_rocky_repo_overrides_post_copy %}{% endblock %} # Install what is needed for en_US.UTF-8 -{% block base_centos_distro_sync_and_languages %} +{% block base_rocky_distro_sync_and_languages %} -{% set base_centos_language_packages = [ +{% set base_rocky_language_packages = [ 'langpacks-en', 'glibc-all-langpacks' ] %} @@ -71,7 +71,7 @@ COPY dnf.conf /etc/dnf/dnf.conf # NOTE(hrw): this macro file drops all languages other than C.UTF-8 so horizon fails # https://bugzilla.redhat.com/show_bug.cgi?id=1729770 RUN rm -f /etc/rpm/macros.image-language-conf \ - && {{ macros.install_packages(base_centos_language_packages | customizable("centos_language_packages"), chain=True, clean=False) }} \ + && {{ macros.install_packages(base_rocky_language_packages | customizable("rocky_language_packages"), chain=True, clean=False) }} \ && {{ macros.rpm_security_update(clean_package_cache) }} {% endblock %} @@ -88,29 +88,29 @@ RUN rm -f /etc/rpm/macros.image-language-conf \ {{ cmd }} {% endfor %} -{% block base_centos_repo_overrides_post_rpm %}{% endblock %} +{% block base_rocky_repo_overrides_post_rpm %}{% endblock %} -{% block base_centos_gpg_key_import %} +{% block base_rocky_gpg_key_import %} {% endblock %} -{% set base_centos_yum_repo_keys = [ +{% set base_rocky_yum_repo_keys = [ ] %} -{% set base_centos_yum_repo_packages = [ +{% set base_rocky_yum_repo_packages = [ 'centos-release-ceph-squid', 'centos-release-nfv-openvswitch', 'epel-release', ] %} # We need 'dnf-plugins-core' for 'dnf config-manager' -{% set base_centos_yum_repo_packages = base_centos_yum_repo_packages + [ +{% set base_rocky_yum_repo_packages = base_rocky_yum_repo_packages + [ 'dnf-plugins-core' ] %} -RUN {{ macros.install_packages(base_centos_yum_repo_packages | customizable("centos_yum_repo_packages"), chain=True, clean=False) }} +RUN {{ macros.install_packages(base_rocky_yum_repo_packages | customizable("rocky_yum_repo_packages"), chain=True, clean=False) }} -{%- for key in base_centos_yum_repo_keys | customizable('centos_yum_repo_keys') %} && rpm --import {{ key }} {% endfor %} \ -{% block base_centos_repo_overrides_post_yum %}{% endblock -%} +{%- for key in base_rocky_yum_repo_keys | customizable('rocky_yum_repo_keys') %} && rpm --import {{ key }} {% endfor %} \ +{% block base_rocky_repo_overrides_post_yum %}{% endblock -%} && {{ macros.rpm_security_update(clean_package_cache) }} #### END REPO ENABLEMENT @@ -120,8 +120,7 @@ RUN {{ macros.install_packages(base_centos_yum_repo_packages | customizable("cen {% block base_redhat_binary_versionlock %}{% endblock %} -{# NOTE(hrw): CentOS Stream 9 has curl-minimal, Rocky Linux 9 has curl so we do not install any #} -{% set base_centos_packages = [ +{% set base_rocky_packages = [ 'ca-certificates', 'crypto-policies-scripts', 'dumb-init', @@ -148,7 +147,7 @@ RUN {{ macros.install_packages(base_centos_yum_repo_packages | customizable("cen # Install base packages {{ macros.enable_extra_repos(['epel']) }} -{{ macros.install_packages(base_centos_packages | customizable("centos_packages") | customizable("centos_binary_packages") | customizable("centos_source_packages")) }} +{{ macros.install_packages(base_rocky_packages | customizable("rocky_packages")) }} {# endif for base_package_type rpm #} {% elif base_package_type == 'deb' %} @@ -302,11 +301,6 @@ RUN apt-get --error-on=any update \ {%- if not loop.last %} \{% endif -%} {%- endfor %} -{% if base_distro == 'centos' %} -RUN sed -ri '/-session(\s+)optional(\s+)pam_systemd.so/d' /etc/pam.d/system-auth \ - && sed -ri '/^[^#]/ s/systemd//g' /etc/nsswitch.conf -{% endif %} - COPY set_configs.py /usr/local/bin/kolla_set_configs COPY start.sh /usr/local/bin/kolla_start COPY copy_cacerts.sh /usr/local/bin/kolla_copy_cacerts @@ -349,4 +343,5 @@ RUN touch /usr/local/bin/kolla_extend_start \ {{ macros.kolla_patch_sources() }} {% block base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} CMD ["kolla_start"] diff --git a/kolla/docker/base/ci-centos.repo b/kolla/docker/base/ci-centos.repo deleted file mode 100644 index 4c08f1fea..000000000 --- a/kolla/docker/base/ci-centos.repo +++ /dev/null @@ -1,75 +0,0 @@ -[baseos] -name=(OpenDev mirror) CentOS Stream $releasever - BaseOS -baseurl=http://MIRROR/centos-stream/$stream/BaseOS/$basearch/os/ -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial -gpgcheck=1 -repo_gpgcheck=0 -enabled=1 - -[appstream] -name=(OpenDev mirror) CentOS Stream $releasever - AppStream -baseurl=http://MIRROR/centos-stream/$stream/AppStream/$basearch/os/ -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial -gpgcheck=1 -repo_gpgcheck=0 -enabled=1 - -[extras-common] -name=(OpenDev mirror) CentOS Stream $releasever - Extras packages -baseurl=http://MIRROR/centos-stream/SIGs/$stream/extras/$basearch/extras-common/ -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA512 -gpgcheck=1 -repo_gpgcheck=0 -enabled=1 - -[crb] -name=(OpenDev mirror) CentOS Stream $releasever - CRB -baseurl=http://MIRROR/centos-stream/$stream/CRB/$basearch/os/ -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial -gpgcheck=1 -repo_gpgcheck=0 -enabled=0 - -[highavailability] -name=(OpenDev mirror) CentOS Stream $releasever - HighAvailability -baseurl=http://MIRROR/centos-stream/$stream/HighAvailability/$basearch/os/ -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial -gpgcheck=1 -repo_gpgcheck=0 -enabled=0 - -[centos-ceph-reef] -name=(OpenDev mirror) CentOS-$stream - Ceph Reef -baseurl=http://MIRROR/centos-stream/SIGs/$stream/storage/$basearch/ceph-reef/ -gpgcheck=1 -enabled=0 -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage - -[centos-nfv-openvswitch] -name=(OpenDev mirror) CentOS Stream $releasever - NFV OpenvSwitch -baseurl=http://MIRROR/centos-stream/SIGs/$stream/nfv/$basearch/openvswitch-2/ -gpgcheck=1 -enabled=0 -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-NFV -module_hotfixes=1 - -[centos-opstools] -name=(OpenDev mirror) CentOS Stream $releasever - OpsTools - collectd -baseurl=http://MIRROR/centos-stream/SIGs/$stream/opstools/$basearch/collectd-5/ -gpgcheck=1 -enabled=0 -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-OpsTools - -[centos-rabbitmq-38] -name=(OpenDev mirror) CentOS-9 - RabbitMQ 38 -baseurl=http://MIRROR/centos-stream/SIGs/$stream/messaging/$basearch/rabbitmq-38 -gpgcheck=1 -enabled=0 -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Messaging - -[epel] -name=(OpenDev mirror) Extra Packages for Enterprise Linux $releasever - $basearch -baseurl=http://MIRROR/epel/$releasever/Everything/$basearch/ -enabled=0 -gpgcheck=1 -gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-$releasever diff --git a/kolla/docker/bifrost/bifrost-base/Dockerfile.j2 b/kolla/docker/bifrost/bifrost-base/Dockerfile.j2 index 3e82a9fd3..75e2fe822 100644 --- a/kolla/docker/bifrost/bifrost-base/Dockerfile.j2 +++ b/kolla/docker/bifrost/bifrost-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set venv_path='/var/lib/kolla/venv' %} {{ macros.configure_user(name='bifrost') }} @@ -18,7 +20,7 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% if base_package_type == 'rpm' %} {# NOTE(mgoddard): EPEL required for nginx #} -{{ macros.enable_extra_repos(['epel']) }} +{{ macros.enable_extra_repos(['crb', 'epel']) }} {% endif %} {{ macros.install_packages(bifrost_base_packages | customizable("packages")) }} @@ -74,3 +76,4 @@ RUN chmod 750 /etc/sudoers.d \ {{ macros.kolla_patch_sources() }} {% block bifrost_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/bifrost/bifrost-deploy/Dockerfile.j2 b/kolla/docker/bifrost/bifrost-deploy/Dockerfile.j2 index 04fc733fc..3cb3d82a8 100644 --- a/kolla/docker/bifrost/bifrost-deploy/Dockerfile.j2 +++ b/kolla/docker/bifrost/bifrost-deploy/Dockerfile.j2 @@ -9,6 +9,8 @@ ENV container docker {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + RUN rm -f $(find /lib/systemd/system/sysinit.target.wants/ ! -name systemd-tmpfiles-setup.service -type l); \ rm -f /lib/systemd/system/multi-user.target.wants/*; \ rm -f /etc/systemd/system/*.wants/*; \ @@ -33,3 +35,4 @@ CMD [ "/sbin/init" ] {% block bifrost_deploy_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/blazar/blazar-base/Dockerfile.j2 b/kolla/docker/blazar/blazar-base/Dockerfile.j2 index 433842e41..926ba66a4 100644 --- a/kolla/docker/blazar/blazar-base/Dockerfile.j2 +++ b/kolla/docker/blazar/blazar-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='blazar') }} ADD blazar-base-archive /blazar-base-source @@ -27,3 +29,4 @@ RUN ln -s blazar-base-source/* blazar \ {{ macros.kolla_patch_sources() }} {% block blazar_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ceilometer/ceilometer-base/Dockerfile.j2 b/kolla/docker/ceilometer/ceilometer-base/Dockerfile.j2 index 18bd7600a..4e3a63d02 100644 --- a/kolla/docker/ceilometer/ceilometer-base/Dockerfile.j2 +++ b/kolla/docker/ceilometer/ceilometer-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='ceilometer') }} {% set ceilometer_base_pip_packages = [ @@ -41,3 +43,4 @@ RUN ln -s ceilometer-base-source/* ceilometer \ {{ macros.kolla_patch_sources() }} {% block ceilometer_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ceilometer/ceilometer-compute/Dockerfile.j2 b/kolla/docker/ceilometer/ceilometer-compute/Dockerfile.j2 index 879c4fb01..629aa5283 100644 --- a/kolla/docker/ceilometer/ceilometer-compute/Dockerfile.j2 +++ b/kolla/docker/ceilometer/ceilometer-compute/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set ceilometer_compute_packages = [ 'python3-libvirt' @@ -28,6 +30,7 @@ RUN {{ macros.install_pip(ceilometer_compute_pip_packages | customizable("pip_pa {% block ceilometer_compute_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} # NOTE(huikang): non-root user does not work with libvirt.sock. # Need configuration in nova-libvirt container to allow diff --git a/kolla/docker/ceilometer/ceilometer-ipmi/Dockerfile.j2 b/kolla/docker/ceilometer/ceilometer-ipmi/Dockerfile.j2 index a767dc842..a86c01bac 100644 --- a/kolla/docker/ceilometer/ceilometer-ipmi/Dockerfile.j2 +++ b/kolla/docker/ceilometer/ceilometer-ipmi/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set ceilometer_ipmi_packages = [ 'ipmitool' @@ -23,5 +25,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block ceilometer_ipmi_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER ceilometer diff --git a/kolla/docker/cinder/cinder-api/Dockerfile.j2 b/kolla/docker/cinder/cinder-api/Dockerfile.j2 index b82fba6d3..64a01e404 100644 --- a/kolla/docker/cinder/cinder-api/Dockerfile.j2 +++ b/kolla/docker/cinder/cinder-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set cinder_api_packages = [ ] %} @@ -20,3 +22,4 @@ RUN chmod 644 /usr/local/bin/kolla_cinder_extend_start {% block cinder_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/cinder/cinder-backup/Dockerfile.j2 b/kolla/docker/cinder/cinder-backup/Dockerfile.j2 index 15be355dd..4f42ddbb6 100644 --- a/kolla/docker/cinder/cinder-backup/Dockerfile.j2 +++ b/kolla/docker/cinder/cinder-backup/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set cinder_backup_packages = [ 'device-mapper-multipath', @@ -29,5 +31,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block cinder_backup_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER cinder diff --git a/kolla/docker/cinder/cinder-base/Dockerfile.j2 b/kolla/docker/cinder/cinder-base/Dockerfile.j2 index bd25d9b18..1b85d92ef 100644 --- a/kolla/docker/cinder/cinder-base/Dockerfile.j2 +++ b/kolla/docker/cinder/cinder-base/Dockerfile.j2 @@ -7,11 +7,13 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='cinder') }} {% if base_package_type == 'rpm' %} -{{ macros.enable_extra_repos(['ceph', 'epel']) }} +{{ macros.enable_extra_repos(['ceph', 'crb', 'epel']) }} {% set cinder_base_packages = [ 'ceph-common', @@ -58,3 +60,4 @@ RUN ln -s cinder-base-source/* cinder \ {{ macros.kolla_patch_sources() }} {% block cinder_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/cinder/cinder-volume/Dockerfile.j2 b/kolla/docker/cinder/cinder-volume/Dockerfile.j2 index 0e4f38a6e..772350e4e 100644 --- a/kolla/docker/cinder/cinder-volume/Dockerfile.j2 +++ b/kolla/docker/cinder/cinder-volume/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set cinder_volume_packages = [ 'lsscsi', @@ -54,5 +56,6 @@ RUN chmod 750 /etc/sudoers.d \ {% block cinder_volume_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER cinder diff --git a/kolla/docker/cloudkitty/cloudkitty-api/Dockerfile.j2 b/kolla/docker/cloudkitty/cloudkitty-api/Dockerfile.j2 index fd3d124f3..65dbbe1f6 100644 --- a/kolla/docker/cloudkitty/cloudkitty-api/Dockerfile.j2 +++ b/kolla/docker/cloudkitty/cloudkitty-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set cloudkitty_api_packages = [ ] %} @@ -19,3 +21,4 @@ RUN chmod 644 /usr/local/bin/kolla_cloudkitty_extend_start {% block cloudkitty_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/cloudkitty/cloudkitty-base/Dockerfile.j2 b/kolla/docker/cloudkitty/cloudkitty-base/Dockerfile.j2 index b545d4adf..11e517673 100644 --- a/kolla/docker/cloudkitty/cloudkitty-base/Dockerfile.j2 +++ b/kolla/docker/cloudkitty/cloudkitty-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='cloudkitty') }} {% set cloudkitty_base_pip_packages = [ @@ -27,3 +29,4 @@ RUN ln -s cloudkitty-base-source/* cloudkitty \ {{ macros.kolla_patch_sources() }} {% block cloudkitty_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/cloudkitty/cloudkitty-processor/Dockerfile.j2 b/kolla/docker/cloudkitty/cloudkitty-processor/Dockerfile.j2 index 33f30fd33..e71b52ebb 100644 --- a/kolla/docker/cloudkitty/cloudkitty-processor/Dockerfile.j2 +++ b/kolla/docker/cloudkitty/cloudkitty-processor/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set cloudkitty_processor_packages = [ ] %} @@ -16,5 +18,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block cloudkitty_processor_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER cloudkitty diff --git a/kolla/docker/collectd/Dockerfile.j2 b/kolla/docker/collectd/Dockerfile.j2 index 32ded377e..4ea5a4c40 100644 --- a/kolla/docker/collectd/Dockerfile.j2 +++ b/kolla/docker/collectd/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='collectd') }} {% set collectd_packages = [ @@ -77,5 +79,6 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start \ {% block collectd_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER collectd diff --git a/kolla/docker/cron/Dockerfile.j2 b/kolla/docker/cron/Dockerfile.j2 index d3a9213ac..8a0aef268 100644 --- a/kolla/docker/cron/Dockerfile.j2 +++ b/kolla/docker/cron/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set cron_packages = [ 'cronie', @@ -33,3 +35,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start {% block cron_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/cyborg/cyborg-agent/Dockerfile.j2 b/kolla/docker/cyborg/cyborg-agent/Dockerfile.j2 index dfc1adc0b..e8b5d9c3c 100644 --- a/kolla/docker/cyborg/cyborg-agent/Dockerfile.j2 +++ b/kolla/docker/cyborg/cyborg-agent/Dockerfile.j2 @@ -6,6 +6,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block cyborg_agent_header %}{% endblock %} {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set cyborg_agent_packages = [ 'pciutils', ] %} @@ -20,3 +22,4 @@ RUN {{ macros.install_pip(cyborg_agent_pip_packages | customizable("pip_packages {% block cyborg_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/cyborg/cyborg-api/Dockerfile.j2 b/kolla/docker/cyborg/cyborg-api/Dockerfile.j2 index d67cdeb28..ba4e6f209 100644 --- a/kolla/docker/cyborg/cyborg-api/Dockerfile.j2 +++ b/kolla/docker/cyborg/cyborg-api/Dockerfile.j2 @@ -6,6 +6,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block cyborg_api_header %}{% endblock %} {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set cyborg_api_packages = [ ] %} {{ macros.install_packages(cyborg_api_packages | customizable("packages")) }} @@ -17,3 +19,4 @@ RUN chmod 644 /usr/local/bin/kolla_cyborg_extend_start {% block cyborg_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/cyborg/cyborg-base/Dockerfile.j2 b/kolla/docker/cyborg/cyborg-base/Dockerfile.j2 index c3f3e602a..95a0eab8c 100644 --- a/kolla/docker/cyborg/cyborg-base/Dockerfile.j2 +++ b/kolla/docker/cyborg/cyborg-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='cyborg') }} ADD cyborg-base-archive /cyborg-base-source @@ -29,3 +31,4 @@ RUN ln -s cyborg-base-source/* cyborg \ {{ macros.kolla_patch_sources() }} {% block cyborg_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/designate/designate-backend-bind9/Dockerfile.j2 b/kolla/docker/designate/designate-backend-bind9/Dockerfile.j2 index cf1277aeb..7fc0e69b5 100644 --- a/kolla/docker/designate/designate-backend-bind9/Dockerfile.j2 +++ b/kolla/docker/designate/designate-backend-bind9/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set designate_backend_bind9_packages = ['bind'] %} @@ -29,3 +31,4 @@ RUN mkdir -p /var/lib/kolla/ /var/lib/{{ designate_backend_bind_name }}/ /run/{{ {% block designate_backend_bind9_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/designate/designate-base/Dockerfile.j2 b/kolla/docker/designate/designate-base/Dockerfile.j2 index 26a0beca1..46836d931 100644 --- a/kolla/docker/designate/designate-base/Dockerfile.j2 +++ b/kolla/docker/designate/designate-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='designate') }} {% set designate_base_pip_packages = [ @@ -31,3 +33,4 @@ RUN ln -s designate-base-source/* designate \ {{ macros.kolla_patch_sources() }} {% block designate_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/designate/designate-worker/Dockerfile.j2 b/kolla/docker/designate/designate-worker/Dockerfile.j2 index b4e190df6..1cfff0c54 100644 --- a/kolla/docker/designate/designate-worker/Dockerfile.j2 +++ b/kolla/docker/designate/designate-worker/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + # The bind9 package here is only to provide the rndc binary. {% if base_package_type == 'rpm' %} {% set designate_worker_packages = [ @@ -23,5 +25,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block designate_worker_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER designate diff --git a/kolla/docker/dnsmasq/Dockerfile.j2 b/kolla/docker/dnsmasq/Dockerfile.j2 index 882613be7..a91b2fca8 100644 --- a/kolla/docker/dnsmasq/Dockerfile.j2 +++ b/kolla/docker/dnsmasq/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set dnsmasq_packages = ['dnsmasq'] %} {{ macros.install_packages(dnsmasq_packages| customizable("packages")) }} @@ -17,3 +19,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start {% block dnsmasq_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/fluentd/Dockerfile.j2 b/kolla/docker/fluentd/Dockerfile.j2 index c63e1b990..68dd71976 100644 --- a/kolla/docker/fluentd/Dockerfile.j2 +++ b/kolla/docker/fluentd/Dockerfile.j2 @@ -10,6 +10,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" fluentd_binary="flue {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.enable_extra_repos(['fluentd']) }} {% if base_package_type == 'rpm' %} @@ -58,5 +60,6 @@ COPY extend_start.sh /usr/local/bin/kolla_extend_start {% block fluentd_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER {{ fluentd_user }} diff --git a/kolla/docker/glance/glance-api/Dockerfile.j2 b/kolla/docker/glance/glance-api/Dockerfile.j2 index e3efbfef3..ca1820631 100644 --- a/kolla/docker/glance/glance-api/Dockerfile.j2 +++ b/kolla/docker/glance/glance-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set glance_api_packages = [ 'lsscsi', @@ -34,5 +36,6 @@ RUN chmod 644 /usr/local/bin/kolla_glance_extend_start {% block glance_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER glance diff --git a/kolla/docker/glance/glance-base/Dockerfile.j2 b/kolla/docker/glance/glance-base/Dockerfile.j2 index 885cc4353..59402e663 100644 --- a/kolla/docker/glance/glance-base/Dockerfile.j2 +++ b/kolla/docker/glance/glance-base/Dockerfile.j2 @@ -7,11 +7,13 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='glance') }} {% if base_package_type == 'rpm' %} -{{ macros.enable_extra_repos(['ceph', 'epel']) }} +{{ macros.enable_extra_repos(['ceph', 'crb', 'epel']) }} {% set glance_base_packages = [ 'python3-rados', @@ -52,3 +54,4 @@ RUN ln -s glance-base-source/* glance \ {{ macros.kolla_patch_sources() }} {% block glance_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/gnocchi/gnocchi-base/Dockerfile.j2 b/kolla/docker/gnocchi/gnocchi-base/Dockerfile.j2 index 91199ac39..abbeb5636 100644 --- a/kolla/docker/gnocchi/gnocchi-base/Dockerfile.j2 +++ b/kolla/docker/gnocchi/gnocchi-base/Dockerfile.j2 @@ -7,11 +7,13 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='gnocchi') }} {% if base_package_type == 'rpm' %} -{{ macros.enable_extra_repos(['ceph', 'epel']) }} +{{ macros.enable_extra_repos(['ceph', 'crb', 'epel']) }} {% set gnocchi_base_packages = [ 'librados2-devel', @@ -54,3 +56,4 @@ RUN ln -s gnocchi-base-source/* gnocchi \ {{ macros.kolla_patch_sources() }} {% block gnocchi_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/grafana/Dockerfile.j2 b/kolla/docker/grafana/Dockerfile.j2 index e401d91b6..a3a2210e5 100644 --- a/kolla/docker/grafana/Dockerfile.j2 +++ b/kolla/docker/grafana/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.enable_extra_repos(['grafana']) }} {% set grafana_packages = ['grafana'] %} @@ -22,11 +24,14 @@ COPY extend_start.sh /usr/local/bin/kolla_extend_start {% block grafana_plugins_install %} -{% set grafana_default_plugins = ['grafana-opensearch-datasource'] %} +{% set grafana_default_plugins = [ + 'grafana-opensearch-datasource', + 'prometheus' +] %} {% set grafana_plugins = grafana_default_plugins | customizable("plugins") %} {% if grafana_plugins | length > 0 %} -RUN {{ (['grafana-cli plugins install %s'] * grafana_plugins | length ) | join(' && ') | format(*grafana_plugins) }} +RUN {{ (['grafana cli plugins install %s'] * grafana_plugins | length ) | join(' && ') | format(*grafana_plugins) }} {% endif %} {% endblock %} @@ -38,5 +43,6 @@ RUN chmod 750 /etc/sudoers.d \ {% block grafana_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER grafana diff --git a/kolla/docker/hacluster/hacluster-base/Dockerfile.j2 b/kolla/docker/hacluster/hacluster-base/Dockerfile.j2 index 045f21e1e..52d255b88 100644 --- a/kolla/docker/hacluster/hacluster-base/Dockerfile.j2 +++ b/kolla/docker/hacluster/hacluster-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type =='rpm' %} {{ macros.enable_extra_repos(['hacluster']) }} @@ -20,3 +22,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start {{ macros.kolla_patch_sources() }} {% block hacluster_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/hacluster/hacluster-corosync/Dockerfile.j2 b/kolla/docker/hacluster/hacluster-corosync/Dockerfile.j2 index 74d47c380..e8ed1f147 100644 --- a/kolla/docker/hacluster/hacluster-corosync/Dockerfile.j2 +++ b/kolla/docker/hacluster/hacluster-corosync/Dockerfile.j2 @@ -7,9 +7,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set hacluster_corosync_packages = ['corosync'] %} {{ macros.install_packages(hacluster_corosync_packages | customizable("packages")) }} {{ macros.kolla_patch_sources() }} {% block hacluster_corosync_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/hacluster/hacluster-pacemaker-remote/Dockerfile.j2 b/kolla/docker/hacluster/hacluster-pacemaker-remote/Dockerfile.j2 index 64c381e1f..a6bf2a7d4 100644 --- a/kolla/docker/hacluster/hacluster-pacemaker-remote/Dockerfile.j2 +++ b/kolla/docker/hacluster/hacluster-pacemaker-remote/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set hacluster_pacemaker_remote_packages = ['pacemaker-remote'] %} {% if base_package_type == 'deb' %} {% set hacluster_pacemaker_remote_packages = hacluster_pacemaker_remote_packages + [ @@ -19,3 +21,4 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {{ macros.kolla_patch_sources() }} {% block hacluster_pacemaker_remote_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/hacluster/hacluster-pacemaker/Dockerfile.j2 b/kolla/docker/hacluster/hacluster-pacemaker/Dockerfile.j2 index 33eeb3144..e3c38fbb8 100644 --- a/kolla/docker/hacluster/hacluster-pacemaker/Dockerfile.j2 +++ b/kolla/docker/hacluster/hacluster-pacemaker/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set hacluster_pacemaker_packages = ['pacemaker'] %} {% if base_package_type == 'deb' %} {% set hacluster_pacemaker_packages = hacluster_pacemaker_packages + [ @@ -19,3 +21,4 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {{ macros.kolla_patch_sources() }} {% block hacluster_pacemaker_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/hacluster/hacluster-pcs/Dockerfile.j2 b/kolla/docker/hacluster/hacluster-pcs/Dockerfile.j2 index a146d0d4b..c744ad68c 100644 --- a/kolla/docker/hacluster/hacluster-pcs/Dockerfile.j2 +++ b/kolla/docker/hacluster/hacluster-pcs/Dockerfile.j2 @@ -7,9 +7,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set hacluster_pcs_packages = ['pcs','crmsh'] %} {{ macros.install_packages(hacluster_pcs_packages | customizable("packages")) }} {{ macros.kolla_patch_sources() }} {% block hacluster_pcs_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/haproxy/haproxy-ssh/Dockerfile.j2 b/kolla/docker/haproxy/haproxy-ssh/Dockerfile.j2 index a674cc75e..dda1c9d6f 100644 --- a/kolla/docker/haproxy/haproxy-ssh/Dockerfile.j2 +++ b/kolla/docker/haproxy/haproxy-ssh/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='haproxy', shell='/bin/bash') }} {% if base_package_type == 'rpm' %} @@ -40,3 +42,4 @@ COPY update-loadbalancer-cert.sh /usr/bin/update-loadbalancer-cert {% block haproxy_ssh_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/haproxy/haproxy/Dockerfile.j2 b/kolla/docker/haproxy/haproxy/Dockerfile.j2 index 36c7ca943..aed0615fe 100644 --- a/kolla/docker/haproxy/haproxy/Dockerfile.j2 +++ b/kolla/docker/haproxy/haproxy/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='haproxy') }} {% set haproxy_packages = [ @@ -24,3 +26,4 @@ RUN mkdir -p /var/lib/kolla/haproxy \ {% block haproxy_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/heat/heat-base/Dockerfile.j2 b/kolla/docker/heat/heat-base/Dockerfile.j2 index a06ea6f0d..65afe31fb 100644 --- a/kolla/docker/heat/heat-base/Dockerfile.j2 +++ b/kolla/docker/heat/heat-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='heat') }} {% if base_package_type == 'rpm' %} @@ -39,3 +41,4 @@ RUN ln -s heat-base-source/* heat \ {{ macros.kolla_patch_sources() }} {% block heat_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/horizon/Dockerfile.j2 b/kolla/docker/horizon/Dockerfile.j2 index 73dc65dd1..6e0e8e84e 100644 --- a/kolla/docker/horizon/Dockerfile.j2 +++ b/kolla/docker/horizon/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='horizon') }} {% if base_package_type == 'rpm' %} @@ -72,3 +74,4 @@ RUN ln -s horizon-source/* horizon \ {% block horizon_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/httpd/Dockerfile.j2 b/kolla/docker/httpd/Dockerfile.j2 index ec5637283..3c86c57b7 100644 --- a/kolla/docker/httpd/Dockerfile.j2 +++ b/kolla/docker/httpd/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {{ macros.enable_extra_repos(['epel']) }} {% set apache_packages = [ @@ -37,6 +39,8 @@ RUN echo > /etc/apache2/ports.conf COPY extend_start.sh /usr/local/bin/kolla_extend_start COPY httpd_setup.sh /usr/local/bin/kolla_httpd_setup RUN chmod 644 /usr/local/bin/kolla_extend_start /usr/local/bin/kolla_httpd_setup +{{ macros.kolla_patch_sources() }} USER root {% block apache_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/httpd/httpd_setup.sh b/kolla/docker/httpd/httpd_setup.sh index 35f7b5b1e..5188c5518 100644 --- a/kolla/docker/httpd/httpd_setup.sh +++ b/kolla/docker/httpd/httpd_setup.sh @@ -17,7 +17,7 @@ if [[ "$(whoami)" == 'root' ]]; then rm -rf /var/run/apache2/* else rm -rf /var/run/httpd/* /run/httpd/* /tmp/httpd* - # NOTE(mmalchuk): This added to make Rocky/Centos similar to Ubuntu/Debian + # NOTE(mmalchuk): This added to make Rocky similar to Ubuntu/Debian # to provide /server-status handler for local monitoring of the Apache. # The module already loaded in the /etc/httpd/conf.modules.d/00-base.conf. cat << EOF >/etc/httpd/conf.modules.d/99-server-status.conf @@ -28,13 +28,13 @@ if [[ "$(whoami)" == 'root' ]]; then EOF fi - # CentOS/Rocky have an issue with mod_ssl which produces an invalid Apache + # Rocky have an issue with mod_ssl which produces an invalid Apache # configuration in /etc/httpd/conf.d/ssl.conf. This causes the following error # on startup: # SSLCertificateFile: file '/etc/pki/tls/certs/localhost.crt' does not exist or is empty # Work around this by generating certificates manually. # NOTE(mnasiadka): in EL9 upgrade jobs gencerts is failing on wrong permissions to dhparams.pem - if [[ "${KOLLA_BASE_DISTRO}" =~ centos|rocky ]] && [[ ! -e /etc/pki/tls/certs/localhost.crt ]]; then + if [[ "${KOLLA_BASE_DISTRO}" =~ rocky ]] && [[ ! -e /etc/pki/tls/certs/localhost.crt ]]; then rm -f /tmp/dhparams.pem /usr/libexec/httpd-ssl-gencerts fi diff --git a/kolla/docker/ironic/ironic-api/Dockerfile.j2 b/kolla/docker/ironic/ironic-api/Dockerfile.j2 index 1c10d0372..f5337f04a 100644 --- a/kolla/docker/ironic/ironic-api/Dockerfile.j2 +++ b/kolla/docker/ironic/ironic-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set ironic_api_packages = [ ] %} {{ macros.install_packages(ironic_api_packages | customizable("packages")) }} @@ -18,3 +20,4 @@ RUN chmod 644 /usr/local/bin/kolla_ironic_extend_start {% block ironic_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ironic/ironic-base/Dockerfile.j2 b/kolla/docker/ironic/ironic-base/Dockerfile.j2 index 39265cf80..7696d2911 100644 --- a/kolla/docker/ironic/ironic-base/Dockerfile.j2 +++ b/kolla/docker/ironic/ironic-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='ironic', shell='/bin/bash') }} ADD ironic-base-archive /ironic-base-source @@ -30,3 +32,4 @@ RUN ln -s ironic-base-source/* ironic \ {{ macros.kolla_patch_sources() }} {% block ironic_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ironic/ironic-conductor/Dockerfile.j2 b/kolla/docker/ironic/ironic-conductor/Dockerfile.j2 index 15a07e4b9..40525b620 100644 --- a/kolla/docker/ironic/ironic-conductor/Dockerfile.j2 +++ b/kolla/docker/ironic/ironic-conductor/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set ironic_conductor_packages = [ 'dosfstools', @@ -71,5 +73,6 @@ RUN {{ macros.install_pip(ironic_conductor_pip_packages | customizable("pip_pack {% block ironic_conductor_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER ironic diff --git a/kolla/docker/ironic/ironic-pxe-filter/Dockerfile.j2 b/kolla/docker/ironic/ironic-pxe-filter/Dockerfile.j2 index 5b593e9f0..717c7849c 100644 --- a/kolla/docker/ironic/ironic-pxe-filter/Dockerfile.j2 +++ b/kolla/docker/ironic/ironic-pxe-filter/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set ironic_pxe_filter_packages = [] %} {% set ironic_pxe_filter_pip_packages = [] %} @@ -22,3 +24,4 @@ RUN chmod 644 /usr/local/bin/kolla_ironic_extend_start {% block ironic_pxe_filter_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ironic/ironic-pxe/Dockerfile.j2 b/kolla/docker/ironic/ironic-pxe/Dockerfile.j2 index 0475a5b05..f198e3292 100644 --- a/kolla/docker/ironic/ironic-pxe/Dockerfile.j2 +++ b/kolla/docker/ironic/ironic-pxe/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set ironic_pxe_packages = [ 'dosfstools', @@ -53,3 +55,4 @@ RUN chmod 644 /usr/local/bin/kolla_ironic_extend_start {% block ironic_pxe_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ironic/ironic-pxe/extend_start.sh b/kolla/docker/ironic/ironic-pxe/extend_start.sh index 06a8312b4..e6100247d 100644 --- a/kolla/docker/ironic/ironic-pxe/extend_start.sh +++ b/kolla/docker/ironic/ironic-pxe/extend_start.sh @@ -7,7 +7,7 @@ function prepare_pxe_pxelinux { cp /usr/lib/PXELINUX/pxelinux.0 \ /usr/lib/syslinux/modules/bios/{chain.c32,ldlinux.c32} \ ${TFTPBOOT_PATH}/ - elif [[ "${KOLLA_BASE_DISTRO}" =~ centos|rocky ]]; then + elif [[ "${KOLLA_BASE_DISTRO}" =~ rocky ]]; then if [[ "${TFTPBOOT_PATH}" != /tftpboot ]]; then cp /tftpboot/{pxelinux.0,chain.c32,ldlinux.c32} \ ${TFTPBOOT_PATH}/ @@ -20,7 +20,7 @@ function prepare_pxe_grub { if [[ "${KOLLA_BASE_DISTRO}" =~ debian|ubuntu ]]; then shim_src_file="/usr/lib/shim/shim*64.efi.signed" grub_src_file="/usr/lib/grub/*-efi-signed/grubnet*64.efi.signed" - elif [[ "${KOLLA_BASE_DISTRO}" =~ centos|rocky ]]; then + elif [[ "${KOLLA_BASE_DISTRO}" =~ rocky ]]; then shim_src_file="/boot/efi/EFI/${KOLLA_BASE_DISTRO}/shim*64.efi" grub_src_file="/boot/efi/EFI/${KOLLA_BASE_DISTRO}/grub*64.efi" fi @@ -46,15 +46,13 @@ function prepare_ipxe { # NOTE(m-anson): ipxe-arm64.efi is not symlinked from /boot to # /usr/lib/ipxe by the Ubuntu ipxe package, so fix that here. if [[ -e /boot/ipxe-arm64.efi ]]; then - ln -s /boot/ipxe-arm64.efi /usr/lib/ipxe/ + ln -sf /boot/ipxe-arm64.efi /usr/lib/ipxe/ fi cp /usr/lib/ipxe/{undionly.kpxe,ipxe*.efi,snponly.efi} ${TFTPBOOT_PATH}/ - elif [[ "${KOLLA_BASE_DISTRO}" =~ centos|rocky ]]; then + elif [[ "${KOLLA_BASE_DISTRO}" =~ rocky ]]; then cp /usr/share/ipxe/{undionly.kpxe,ipxe-snponly-x86_64.efi} ${TFTPBOOT_PATH}/ cp /usr/share/ipxe/arm64-efi/snponly.efi ${TFTPBOOT_PATH}/ipxe-snponly-aarch64.efi - if [[ ! -e ${TFTPBOOT_PATH}/snponly.efi ]]; then - ln -s ${TFTPBOOT_PATH}/ipxe-snponly-${KOLLA_BASE_ARCH}.efi ${TFTPBOOT_PATH}/snponly.efi - fi + ln -sf ${TFTPBOOT_PATH}/ipxe-snponly-${KOLLA_BASE_ARCH}.efi ${TFTPBOOT_PATH}/snponly.efi fi } @@ -65,7 +63,7 @@ function prepare_esp_image { if [[ "${KOLLA_BASE_DISTRO}" =~ debian|ubuntu ]]; then shim_src_file="/usr/lib/shim/shim*64.efi.signed" grub_src_file="/usr/lib/grub/*-efi-signed/grubnet*64.efi.signed" - elif [[ "${KOLLA_BASE_DISTRO}" =~ centos|rocky ]]; then + elif [[ "${KOLLA_BASE_DISTRO}" =~ rocky ]]; then shim_src_file="/boot/efi/EFI/${KOLLA_BASE_DISTRO}/shim*64.efi" grub_src_file="/boot/efi/EFI/${KOLLA_BASE_DISTRO}/grub*64.efi" fi diff --git a/kolla/docker/iscsid/Dockerfile.j2 b/kolla/docker/iscsid/Dockerfile.j2 index a1e4bba61..d4f41463c 100644 --- a/kolla/docker/iscsid/Dockerfile.j2 +++ b/kolla/docker/iscsid/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set iscsid_packages = [ 'iscsi-initiator-utils', @@ -26,7 +28,7 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {{ macros.install_packages(iscsid_packages | customizable("packages")) }} COPY extend_start.sh /usr/local/bin/kolla_extend_start -# NOTE(hrw): CentOS Stream 9 lacks initiatorname.iscsi file +# NOTE(hrw): Rocky Linux lacks initiatorname.iscsi file RUN chmod 644 /usr/local/bin/kolla_extend_start \ && rm -f /etc/iscsi/initiatorname.iscsi @@ -34,3 +36,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start \ {% block iscsid_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/keepalived/Dockerfile.j2 b/kolla/docker/keepalived/Dockerfile.j2 index 2dcdeccd7..93b8218c8 100644 --- a/kolla/docker/keepalived/Dockerfile.j2 +++ b/kolla/docker/keepalived/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.enable_extra_repos(['mariadb']) }} {% if base_package_type == 'rpm' %} @@ -32,3 +34,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start \ {% block keepalived_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/kerbside/kerbside-api/Dockerfile.j2 b/kolla/docker/kerbside/kerbside-api/Dockerfile.j2 index 6ea99eb7a..28557fe52 100644 --- a/kolla/docker/kerbside/kerbside-api/Dockerfile.j2 +++ b/kolla/docker/kerbside/kerbside-api/Dockerfile.j2 @@ -3,7 +3,9 @@ FROM {{ namespace }}/{{ image_prefix }}kerbside-base:{{ tag }} LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% endblock %} +{% import "macros.j2" as macros with context %} + COPY extend_start.sh /usr/local/bin/kolla_extend_start RUN chmod 644 /usr/local/bin/kolla_extend_start - +{{ macros.kolla_patch_sources() }} USER kerbside diff --git a/kolla/docker/kerbside/kerbside-base/Dockerfile.j2 b/kolla/docker/kerbside/kerbside-base/Dockerfile.j2 index 77f881399..bc181258f 100644 --- a/kolla/docker/kerbside/kerbside-base/Dockerfile.j2 +++ b/kolla/docker/kerbside/kerbside-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='kerbside', groups='kerbside') }} {% if base_package_type == 'rpm' %} @@ -35,4 +37,6 @@ RUN ln -s kerbside-source/* kerbside \ && {{ macros.install_pip(kerbside_pip_packages | customizable("pip_packages"), false) }} \ && rm -f /etc/machine-id +{{ macros.kolla_patch_sources() }} {% block kerbside_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/kerbside/kerbside-proxy/Dockerfile.j2 b/kolla/docker/kerbside/kerbside-proxy/Dockerfile.j2 index 6ea99eb7a..28557fe52 100644 --- a/kolla/docker/kerbside/kerbside-proxy/Dockerfile.j2 +++ b/kolla/docker/kerbside/kerbside-proxy/Dockerfile.j2 @@ -3,7 +3,9 @@ FROM {{ namespace }}/{{ image_prefix }}kerbside-base:{{ tag }} LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% endblock %} +{% import "macros.j2" as macros with context %} + COPY extend_start.sh /usr/local/bin/kolla_extend_start RUN chmod 644 /usr/local/bin/kolla_extend_start - +{{ macros.kolla_patch_sources() }} USER kerbside diff --git a/kolla/docker/keystone/keystone-base/Dockerfile.j2 b/kolla/docker/keystone/keystone-base/Dockerfile.j2 index faa424fa7..82c90e6f6 100644 --- a/kolla/docker/keystone/keystone-base/Dockerfile.j2 +++ b/kolla/docker/keystone/keystone-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='keystone') }} {% if base_package_type == 'rpm' %} @@ -43,3 +45,4 @@ RUN ln -s keystone-base-source/* keystone \ {{ macros.kolla_patch_sources() }} {% block keystone_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/keystone/keystone-fernet/Dockerfile.j2 b/kolla/docker/keystone/keystone-fernet/Dockerfile.j2 index da32ddb86..92ee4738b 100644 --- a/kolla/docker/keystone/keystone-fernet/Dockerfile.j2 +++ b/kolla/docker/keystone/keystone-fernet/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set keystone_fernet_packages = [ 'cronie', @@ -32,3 +34,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start \ {% block keystone_fernet_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/keystone/keystone-ssh/Dockerfile.j2 b/kolla/docker/keystone/keystone-ssh/Dockerfile.j2 index af290bd5f..a7b2a6e10 100644 --- a/kolla/docker/keystone/keystone-ssh/Dockerfile.j2 +++ b/kolla/docker/keystone/keystone-ssh/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='keystone', shell='/bin/bash') }} {% if base_package_type == 'rpm' %} @@ -36,3 +38,4 @@ RUN sed -ri 's/session(\s+)required(\s+)pam_loginuid.so/session\1optional\2pam_l {% block keystone_ssh_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/keystone/keystone-ssh/extend_start.sh b/kolla/docker/keystone/keystone-ssh/extend_start.sh index 559cd65a9..a6418b4d9 100644 --- a/kolla/docker/keystone/keystone-ssh/extend_start.sh +++ b/kolla/docker/keystone/keystone-ssh/extend_start.sh @@ -1,6 +1,6 @@ #!/bin/bash -# NOTE(mnasiadka): CentOS 10 does not support dsa +# NOTE(mnasiadka): Rocky 10 does not support dsa SSH_HOST_KEY_TYPES=( "rsa" "ecdsa" "ed25519" ) for key_type in ${SSH_HOST_KEY_TYPES[@]}; do diff --git a/kolla/docker/keystone/keystone/Dockerfile.j2 b/kolla/docker/keystone/keystone/Dockerfile.j2 index 742626c05..df2003f10 100644 --- a/kolla/docker/keystone/keystone/Dockerfile.j2 +++ b/kolla/docker/keystone/keystone/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set keystone_packages = [ 'krb5-devel', @@ -43,3 +45,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start \ {% block keystone_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/kolla-toolbox/Dockerfile.j2 b/kolla/docker/kolla-toolbox/Dockerfile.j2 index cad44524c..2e4750e2a 100644 --- a/kolla/docker/kolla-toolbox/Dockerfile.j2 +++ b/kolla/docker/kolla-toolbox/Dockerfile.j2 @@ -10,6 +10,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='ansible') }} {{ macros.configure_user(name='rabbitmq') }} @@ -125,5 +127,6 @@ RUN chmod 644 /etc/ansible/ansible.cfg \ {% block kolla_toolbox_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER ansible diff --git a/kolla/docker/letsencrypt/letsencrypt-lego/Dockerfile.j2 b/kolla/docker/letsencrypt/letsencrypt-lego/Dockerfile.j2 index b7cda214e..a52199d0b 100755 --- a/kolla/docker/letsencrypt/letsencrypt-lego/Dockerfile.j2 +++ b/kolla/docker/letsencrypt/letsencrypt-lego/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='letsencrypt', shell='/bin/bash') }} {% if base_package_type == 'rpm' %} @@ -39,3 +41,4 @@ RUN chmod +x /usr/bin/letsencrypt-certificates /usr/bin/sync-and-update-certific {% block letsencrypt_lego_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/letsencrypt/letsencrypt-lego/letsencrypt-certificates.sh b/kolla/docker/letsencrypt/letsencrypt-lego/letsencrypt-certificates.sh index 83bdede87..4c06688e1 100755 --- a/kolla/docker/letsencrypt/letsencrypt-lego/letsencrypt-certificates.sh +++ b/kolla/docker/letsencrypt/letsencrypt-lego/letsencrypt-certificates.sh @@ -24,6 +24,9 @@ function obtain_or_renew_certificate { local hmac="${9}" local key_id="${10}" local letsencrypt_key_type="${11:-}" + local challenge_type="${12:-http-01}" + local dns_provider="${13:-}" + local dns_additional_opts="${14:-}" certificate_domain_opts=$(echo ${certificate_fqdns} | sed -r -e 's/^/,/g' -e 's/,/--domains=/g' -e 's/--/ --/g') certificate_fqdn=$(echo ${certificate_fqdns} | awk -F ',' '{print $1}') @@ -68,17 +71,24 @@ function obtain_or_renew_certificate { fi fi - log_info "[${certificate_fqdn} - cron] Obtaining certificate for domains ${certificate_fqdns}." + if [ "${challenge_type}" = "http-01" ]; then + challenge_opts="--http --http.webroot /etc/letsencrypt/http-01 --http.port ${listen_port}" + elif [ "${challenge_type}" = "dns-01" ]; then + challenge_opts="--dns ${dns_provider} ${dns_additional_opts}" + else + log_error "Unsupported challenge type: ${challenge_type}." + exit 1 + fi + + log_info "[${certificate_fqdn} - cron] Obtaining certificate for domains ${certificate_fqdns} using ${challenge_type} challenge." mapfile -t cmd_output < <(/opt/lego --email="${mail}" \ $( [ -n "${letsencrypt_key_type}" ] && echo "--key-type ${letsencrypt_key_type}" ) \ ${certificate_domain_opts} \ --server "${acme_url}" \ --path "/etc/letsencrypt/lego/${certificate_type}/" \ - --http.webroot "/etc/letsencrypt/http-01" \ - --http.port ${listen_port} \ + ${challenge_opts} \ --cert.timeout ${valid_days} \ --accept-tos \ - --http \ ${eab_opts} \ --pem ${lego_action} \ --${lego_action}-hook="/usr/bin/sync-and-update-certificate --${certificate_type} --fqdn ${certificate_fqdn} --haproxies-ssh ${letsencrypt_ssh_port}" 2>&1) @@ -117,10 +127,14 @@ EXTERNAL_SET="false" EXTERNAL_ACCOUNT_BINDING="false" HMAC="NONE" KEY_ID="NONE" +CHALLENGE_TYPE="http-01" +DNS_PROVIDER="" +DNS_ADDITIONAL_OPTS="" +DNS_ENV_FILE="" LOG_FILE="/var/log/kolla/letsencrypt/lesencrypt-lego.log" -VALID_ARGS=$(getopt -o ief:p:d:m:a:v:h:k: --long internal,external,fqdns:,port:,days:,mail:,acme:,vips:,haproxies-ssh:,eab,kid:,hmac:,key-type: -- "$@") +VALID_ARGS=$(getopt -o ief:p:d:m:a:v:h:k: --long internal,external,fqdns:,port:,days:,mail:,acme:,vips:,haproxies-ssh:,eab,kid:,hmac:,key-type:,challenge:,dns-provider:,dns-additional-opts:,env-file: -- "$@") if [[ $? -ne 0 ]]; then exit 1; fi @@ -182,12 +196,53 @@ while [ : ]; do LETSENCRYPT_KEY_TYPE="${2}" shift 2 ;; + --challenge) + CHALLENGE_TYPE="${2}" + shift 2 + ;; + --dns-provider) + DNS_PROVIDER="${2}" + shift 2 + ;; + --dns-additional-opts) + DNS_ADDITIONAL_OPTS="${2}" + shift 2 + ;; + --env-file) + DNS_ENV_FILE="${2}" + shift 2 + ;; --) shift; break ;; esac done +if [ "${CHALLENGE_TYPE}" = "dns-01" ]; then + if [ "${DNS_PROVIDER}" = "" ]; then + log_error "dns-01 challenge requires --dns-provider." + exit 1 + fi + # lego DNS providers take their credentials from environment variables, + # so the env file must be exported into this process before lego runs. + # The file is parsed like docker --env-file: literal KEY=VALUE pairs, + # split on the first '=', values taken verbatim. It is deliberately not + # sourced, so values containing spaces or shell metacharacters survive + # and nothing in the file is ever executed. + if [ "${DNS_ENV_FILE}" != "" ]; then + if [ ! -r "${DNS_ENV_FILE}" ]; then + log_error "DNS provider environment file ${DNS_ENV_FILE} is not readable." + exit 1 + fi + while IFS='=' read -r env_key env_value || [ -n "${env_key}" ]; do + case "${env_key}" in + ''|\#*) continue ;; + esac + export "${env_key}=${env_value}" + done < "${DNS_ENV_FILE}" + fi +fi + FQDN=$(echo "${FQDNS}" | awk -F ',' '{print $1}') if [ "${INTERNAL_SET}" = "true" ] || [ "${EXTERNAL_SET}" = "true" ]; then @@ -209,12 +264,12 @@ if [ "${INTERNAL_SET}" = "true" ] || [ "${EXTERNAL_SET}" = "true" ]; then log_info "[${FQDN} - cron] This Letsencrypt-lego host is active..." if [ "${LETSENCRYPT_INTERNAL_FQDNS}" != "" ]; then log_info "[${FQDN} - cron] Processing domains ${LETSENCRYPT_INTERNAL_FQDNS}" - obtain_or_renew_certificate ${LETSENCRYPT_INTERNAL_FQDNS} internal ${PORT} ${DAYS} ${ACME} ${MAIL} ${LETSENCRYPT_SSH_PORT} ${EXTERNAL_ACCOUNT_BINDING} ${HMAC} ${KEY_ID} ${LETSENCRYPT_KEY_TYPE} + obtain_or_renew_certificate "${LETSENCRYPT_INTERNAL_FQDNS}" "internal" "${PORT}" "${DAYS}" "${ACME}" "${MAIL}" "${LETSENCRYPT_SSH_PORT}" "${EXTERNAL_ACCOUNT_BINDING}" "${HMAC}" "${KEY_ID}" "${LETSENCRYPT_KEY_TYPE}" "${CHALLENGE_TYPE}" "${DNS_PROVIDER}" "${DNS_ADDITIONAL_OPTS}" fi if [ "${LETSENCRYPT_EXTERNAL_FQDNS}" != "" ]; then log_info "[${FQDN} - cron] Processing domains ${LETSENCRYPT_EXTERNAL_FQDNS}" - obtain_or_renew_certificate ${LETSENCRYPT_EXTERNAL_FQDNS} external ${PORT} ${DAYS} ${ACME} ${MAIL} ${LETSENCRYPT_SSH_PORT} ${EXTERNAL_ACCOUNT_BINDING} ${HMAC} ${KEY_ID} ${LETSENCRYPT_KEY_TYPE} + obtain_or_renew_certificate "${LETSENCRYPT_EXTERNAL_FQDNS}" "external" "${PORT}" "${DAYS}" "${ACME}" "${MAIL}" "${LETSENCRYPT_SSH_PORT}" "${EXTERNAL_ACCOUNT_BINDING}" "${HMAC}" "${KEY_ID}" "${LETSENCRYPT_KEY_TYPE}" "${CHALLENGE_TYPE}" "${DNS_PROVIDER}" "${DNS_ADDITIONAL_OPTS}" fi else log_info "[${FQDN} - cron] This Letsencrypt-lego host is passive, nothing to do..." diff --git a/kolla/docker/macros.j2 b/kolla/docker/macros.j2 index 34d67335c..6d43b4ccf 100644 --- a/kolla/docker/macros.j2 +++ b/kolla/docker/macros.j2 @@ -95,3 +95,19 @@ sed -e "s/^{{package_name}}==={{ from }}/{{package_name}}=={{ to }}/g" -i requir RUN kolla_patch {% endif %} {%- endmacro %} + +{% macro kolla_cleanup_repos() -%} +{{ cleanup_repos() }} +{%- endmacro %} + +{% macro kolla_enable_base_repos() %} +{% if base_package_type == 'rpm' %} +{{ handle_repos(['baseos', 'appstream'], 'enable') }} +{% elif base_package_type == 'deb' %} +{% if base_distro == 'debian' %} +{{ handle_repos(['debian', 'debian-security'], 'enable') }} +{% elif base_distro == 'ubuntu' %} +{{ handle_repos(['ubuntu', 'ubuntu-security', 'ubuntu-cloud-archive'], 'enable') }} +{% endif %} +{% endif %} +{% endmacro %} diff --git a/kolla/docker/magnum/magnum-base/Dockerfile.j2 b/kolla/docker/magnum/magnum-base/Dockerfile.j2 index ac17b94b4..ef070f88f 100644 --- a/kolla/docker/magnum/magnum-base/Dockerfile.j2 +++ b/kolla/docker/magnum/magnum-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='magnum') }} ADD magnum-base-archive /magnum-base-source @@ -29,3 +31,4 @@ RUN ln -s magnum-base-source/* magnum \ {{ macros.kolla_patch_sources() }} {% block magnum_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/manila/manila-api/Dockerfile.j2 b/kolla/docker/manila/manila-api/Dockerfile.j2 index d7b75d388..ea5f46195 100644 --- a/kolla/docker/manila/manila-api/Dockerfile.j2 +++ b/kolla/docker/manila/manila-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set manila_api_packages = [ ] %} {{ macros.install_packages(manila_api_packages | customizable("packages")) }} @@ -19,5 +21,6 @@ RUN chmod 644 /usr/local/bin/kolla_manila_extend_start {% block manila_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER manila diff --git a/kolla/docker/manila/manila-base/Dockerfile.j2 b/kolla/docker/manila/manila-base/Dockerfile.j2 index a236a20df..fff2d5bd3 100644 --- a/kolla/docker/manila/manila-base/Dockerfile.j2 +++ b/kolla/docker/manila/manila-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='manila') }} {% if base_package_type == 'rpm' %} @@ -46,3 +48,4 @@ RUN ln -s manila-base-source/* manila \ {{ macros.kolla_patch_sources() }} {% block manila_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/manila/manila-data/Dockerfile.j2 b/kolla/docker/manila/manila-data/Dockerfile.j2 index 06e1d9dc9..f0454c122 100644 --- a/kolla/docker/manila/manila-data/Dockerfile.j2 +++ b/kolla/docker/manila/manila-data/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set manila_data_packages = [ @@ -27,5 +29,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block manila_data_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER manila diff --git a/kolla/docker/manila/manila-share/Dockerfile.j2 b/kolla/docker/manila/manila-share/Dockerfile.j2 index 161b6865a..f5bc29bb7 100644 --- a/kolla/docker/manila/manila-share/Dockerfile.j2 +++ b/kolla/docker/manila/manila-share/Dockerfile.j2 @@ -7,7 +7,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} + +{{ macros.enable_extra_repos(['crb']) }} + {% set manila_share_packages = [ 'ceph-common', 'sqlite', @@ -30,5 +35,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block manila_share_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER manila diff --git a/kolla/docker/mariadb-server/Dockerfile.j2 b/kolla/docker/mariadb-server/Dockerfile.j2 index 612b37330..a9a84bf73 100644 --- a/kolla/docker/mariadb-server/Dockerfile.j2 +++ b/kolla/docker/mariadb-server/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='mysql') }} {% if base_package_type == 'rpm' %} @@ -92,5 +94,6 @@ CMD ["kolla_start"] {% block mariadb_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER mysql diff --git a/kolla/docker/masakari/masakari-base/Dockerfile.j2 b/kolla/docker/masakari/masakari-base/Dockerfile.j2 index 97e31247e..92ac9bad9 100644 --- a/kolla/docker/masakari/masakari-base/Dockerfile.j2 +++ b/kolla/docker/masakari/masakari-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='masakari') }} {% if base_package_type == 'rpm' %} @@ -41,3 +43,4 @@ RUN ln -s masakari-base-source/* masakari \ {{ macros.kolla_patch_sources() }} {% block masakari_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/masakari/masakari-monitors/Dockerfile.j2 b/kolla/docker/masakari/masakari-monitors/Dockerfile.j2 index 92b8c3947..701521b4b 100644 --- a/kolla/docker/masakari/masakari-monitors/Dockerfile.j2 +++ b/kolla/docker/masakari/masakari-monitors/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {{ macros.enable_extra_repos(['hacluster']) }} @@ -51,5 +53,6 @@ RUN ln -s masakari-monitors-source/* masakari-monitors \ {% block masakari_monitors_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER masakari diff --git a/kolla/docker/memcached/Dockerfile.j2 b/kolla/docker/memcached/Dockerfile.j2 index dfe90b18e..8cf2e09d7 100644 --- a/kolla/docker/memcached/Dockerfile.j2 +++ b/kolla/docker/memcached/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='memcached', homedir='/run/memcache') }} {% set memcached_packages = ['memcached'] %} @@ -16,5 +18,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block memcached_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER memcached diff --git a/kolla/docker/mistral/mistral-api/Dockerfile.j2 b/kolla/docker/mistral/mistral-api/Dockerfile.j2 index be3e431e1..0ad4fb767 100644 --- a/kolla/docker/mistral/mistral-api/Dockerfile.j2 +++ b/kolla/docker/mistral/mistral-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set mistral_api_packages = [ ] %} {{ macros.install_packages(mistral_api_packages | customizable("packages")) }} @@ -18,3 +20,4 @@ RUN chmod 644 /usr/local/bin/kolla_mistral_extend_start {% block mistral_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/mistral/mistral-base/Dockerfile.j2 b/kolla/docker/mistral/mistral-base/Dockerfile.j2 index 1d72c1cac..d459fda1f 100644 --- a/kolla/docker/mistral/mistral-base/Dockerfile.j2 +++ b/kolla/docker/mistral/mistral-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='mistral') }} ADD mistral-base-archive /mistral-base-source @@ -36,3 +38,4 @@ RUN ln -s mistral-base-source/* mistral \ {{ macros.kolla_patch_sources() }} {% block mistral_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/multipathd/Dockerfile.j2 b/kolla/docker/multipathd/Dockerfile.j2 index 1341c62a7..19ef76fa5 100644 --- a/kolla/docker/multipathd/Dockerfile.j2 +++ b/kolla/docker/multipathd/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set multipathd_packages = ['device-mapper-multipath'] %} {% elif base_package_type == 'deb' %} @@ -19,3 +21,4 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block multipathd_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/neutron/neutron-base/Dockerfile.j2 b/kolla/docker/neutron/neutron-base/Dockerfile.j2 index 1550a0f76..1c454e0ce 100644 --- a/kolla/docker/neutron/neutron-base/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.enable_extra_repos(['docker-ce']) }} {{ macros.configure_user(name='neutron') }} @@ -101,3 +103,4 @@ RUN ln -s neutron-base-source/* neutron \ {{ macros.kolla_patch_sources() }} {% block neutron_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/neutron/neutron-base/extend_start.sh b/kolla/docker/neutron/neutron-base/extend_start.sh index e2a2dfca4..6e6288ffb 100644 --- a/kolla/docker/neutron/neutron-base/extend_start.sh +++ b/kolla/docker/neutron/neutron-base/extend_start.sh @@ -14,7 +14,7 @@ fi # so no need to even use u-a on RHEL 9 family as there is one provider # (and there is no u-a for ip6tables so script fails) -if [[ ! ${KOLLA_BASE_DISTRO} =~ centos|rocky ]]; then +if [[ ! ${KOLLA_BASE_DISTRO} =~ rocky ]]; then if /usr/bin/update-alternatives --display iptables; then # NOTE(yoctozepto): Kolla-Ansible does not always set KOLLA_LEGACY_IPTABLES; # the workaround below ensures it gets set to `false` in such cases to fix diff --git a/kolla/docker/neutron/neutron-bgp-dragent/Dockerfile.j2 b/kolla/docker/neutron/neutron-bgp-dragent/Dockerfile.j2 index 3ad2ab596..66ee77f63 100644 --- a/kolla/docker/neutron/neutron-bgp-dragent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-bgp-dragent/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + ADD neutron-bgp-dragent-archive /neutron-bgp-dragent-source {% set neutron_bgp_dragent_pip_packages = [ @@ -20,5 +22,6 @@ RUN ln -s neutron-bgp-dragent-source/* neutron_dynamic_routing \ {% block neutron_bgp_dragent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-dhcp-agent/Dockerfile.j2 b/kolla/docker/neutron/neutron-dhcp-agent/Dockerfile.j2 index 035433bf2..750526097 100644 --- a/kolla/docker/neutron/neutron-dhcp-agent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-dhcp-agent/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set neutron_dhcp_agent_packages = [ ] %} @@ -18,5 +20,6 @@ COPY extend_start.sh /usr/local/bin/kolla_extend_start {% block neutron_dhcp_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-l3-agent/Dockerfile.j2 b/kolla/docker/neutron/neutron-l3-agent/Dockerfile.j2 index ae0457c18..52b63462a 100644 --- a/kolla/docker/neutron/neutron-l3-agent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-l3-agent/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set neutron_l3_agent_packages = [ 'libreswan', @@ -28,5 +30,6 @@ COPY extend_start.sh /usr/local/bin/kolla_extend_start {% block neutron_l3_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-l3-agent/extend_start.sh b/kolla/docker/neutron/neutron-l3-agent/extend_start.sh index f95c1eda2..4c7cd345f 100644 --- a/kolla/docker/neutron/neutron-l3-agent/extend_start.sh +++ b/kolla/docker/neutron/neutron-l3-agent/extend_start.sh @@ -12,7 +12,7 @@ fi # nftables framework." # so no need to even use u-a on RHEL 9 family as there is one provider # (and there is no u-a for ip6tables so script fails) -if [[ ! ${KOLLA_BASE_DISTRO} =~ centos|rocky ]]; then +if [[ ! ${KOLLA_BASE_DISTRO} =~ rocky ]]; then if /usr/bin/update-alternatives --display iptables; then # NOTE(yoctozepto): Kolla-Ansible does not always set KOLLA_LEGACY_IPTABLES; # the workaround below ensures it gets set to `false` in such cases to fix diff --git a/kolla/docker/neutron/neutron-metadata-agent/Dockerfile.j2 b/kolla/docker/neutron/neutron-metadata-agent/Dockerfile.j2 index 3736f70c8..b6be8d739 100644 --- a/kolla/docker/neutron/neutron-metadata-agent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-metadata-agent/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set neutron_metadata_agent_packages = [ ] %} @@ -18,5 +20,6 @@ COPY extend_start.sh /usr/local/bin/kolla_extend_start {% block neutron_metadata_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-mlnx-agent/Dockerfile.j2 b/kolla/docker/neutron/neutron-mlnx-agent/Dockerfile.j2 index 4ee89c19a..7eb6305fe 100644 --- a/kolla/docker/neutron/neutron-mlnx-agent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-mlnx-agent/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set neutron_mlnx_agent_packages = [ 'python3-libvirt', ] %} @@ -23,5 +25,6 @@ RUN {{ macros.install_pip(neutron_mlnx_agent_pip_packages | customizable("pip_pa {% block neutron_mlnx_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-openvswitch-agent/Dockerfile.j2 b/kolla/docker/neutron/neutron-openvswitch-agent/Dockerfile.j2 index b9499e6f7..d0a963259 100644 --- a/kolla/docker/neutron/neutron-openvswitch-agent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-openvswitch-agent/Dockerfile.j2 @@ -7,11 +7,14 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.install_packages(neutron_openvswitch_agent_packages | customizable("packages")) }} {{ macros.kolla_patch_sources() }} {% block neutron_openvswitch_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-ovn-agent/Dockerfile.j2 b/kolla/docker/neutron/neutron-ovn-agent/Dockerfile.j2 index fe4e352ac..915bef108 100644 --- a/kolla/docker/neutron/neutron-ovn-agent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-ovn-agent/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set neutron_ovn_agent_packages = [ ] %} @@ -16,5 +18,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block neutron_ovn_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-ovn-vpn-agent/Dockerfile.j2 b/kolla/docker/neutron/neutron-ovn-vpn-agent/Dockerfile.j2 index 4fbaaa78f..ff1f6b48f 100644 --- a/kolla/docker/neutron/neutron-ovn-vpn-agent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-ovn-vpn-agent/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + # Note: VPN is currently not working on RHEL 10 due to a known issue: # https://bugs.launchpad.net/neutron/+bug/2146308 {% if base_package_type == 'rpm' %} @@ -27,5 +29,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block neutron_ovn_vpn_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-server/Dockerfile.j2 b/kolla/docker/neutron/neutron-server/Dockerfile.j2 index bda0d3dcf..0ee7ceabf 100644 --- a/kolla/docker/neutron/neutron-server/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-server/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set neutron_server_pip_packages = [ 'networking-hyperv' ] %} @@ -29,5 +31,6 @@ RUN {{ macros.install_pip(neutron_server_pip_packages | customizable("pip_packag {% block neutron_server_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/neutron/neutron-sriov-agent/Dockerfile.j2 b/kolla/docker/neutron/neutron-sriov-agent/Dockerfile.j2 index c5d74fbb2..c4a1f1d4b 100644 --- a/kolla/docker/neutron/neutron-sriov-agent/Dockerfile.j2 +++ b/kolla/docker/neutron/neutron-sriov-agent/Dockerfile.j2 @@ -7,11 +7,14 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.install_packages(neutron_sriov_agent_packages | customizable("packages")) }} {{ macros.kolla_patch_sources() }} {% block neutron_sriov_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER neutron diff --git a/kolla/docker/nova/nova-api/Dockerfile.j2 b/kolla/docker/nova/nova-api/Dockerfile.j2 index 57846af79..08353a708 100644 --- a/kolla/docker/nova/nova-api/Dockerfile.j2 +++ b/kolla/docker/nova/nova-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set nova_api_packages = [ ] %} {{ macros.install_packages(nova_api_packages | customizable("packages")) }} @@ -18,3 +20,4 @@ RUN chmod 644 /usr/local/bin/kolla_nova_extend_start {% block nova_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/nova/nova-base/Dockerfile.j2 b/kolla/docker/nova/nova-base/Dockerfile.j2 index 423a9760a..da5340f2a 100644 --- a/kolla/docker/nova/nova-base/Dockerfile.j2 +++ b/kolla/docker/nova/nova-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='nova', groups='qemu') }} {% if base_package_type == 'rpm' %} @@ -84,3 +86,4 @@ RUN ln -s nova-base-source/* nova \ {{ macros.kolla_patch_sources() }} {% block nova_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/nova/nova-compute-ironic/Dockerfile.j2 b/kolla/docker/nova/nova-compute-ironic/Dockerfile.j2 index 3c455e2ec..69227056c 100644 --- a/kolla/docker/nova/nova-compute-ironic/Dockerfile.j2 +++ b/kolla/docker/nova/nova-compute-ironic/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set nova_compute_ironic_packages = [ 'genisoimage', 'nvme-cli', @@ -19,5 +21,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block nova_compute_ironic_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER nova diff --git a/kolla/docker/nova/nova-compute/Dockerfile.j2 b/kolla/docker/nova/nova-compute/Dockerfile.j2 index 8b3c1bcfe..864222f11 100644 --- a/kolla/docker/nova/nova-compute/Dockerfile.j2 +++ b/kolla/docker/nova/nova-compute/Dockerfile.j2 @@ -7,9 +7,11 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} -{{ macros.enable_extra_repos(['ceph', 'epel']) }} +{{ macros.enable_extra_repos(['ceph', 'crb', 'epel']) }} {% set nova_compute_packages = [ 'ceph-common', @@ -36,7 +38,7 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" 'xfsprogs' ] %} - # NOTE(hrw): daxio is part of pmdk which is x86-64 only in CentOS 8 + # NOTE(hrw): daxio is part of pmdk which is x86-64 only in Rocky {% if base_arch == 'x86_64' %} {% set nova_compute_packages = nova_compute_packages + [ 'daxio' @@ -108,5 +110,6 @@ RUN {{ macros.install_pip(nova_compute_pip_packages | customizable("pip_packages {% block nova_compute_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER nova diff --git a/kolla/docker/nova/nova-libvirt-spice/Dockerfile.j2 b/kolla/docker/nova/nova-libvirt-spice/Dockerfile.j2 index 4c7ded894..4691df8fd 100644 --- a/kolla/docker/nova/nova-libvirt-spice/Dockerfile.j2 +++ b/kolla/docker/nova/nova-libvirt-spice/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" build-date="{{ build {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='nova', groups='qemu') }} {% if base_package_type == 'rpm' %} @@ -118,3 +120,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start && chmod 744 /usr/local/bin/koll {% block nova_libvirt_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/nova/nova-libvirt/Dockerfile.j2 b/kolla/docker/nova/nova-libvirt/Dockerfile.j2 index fb82401b3..cfca7306a 100644 --- a/kolla/docker/nova/nova-libvirt/Dockerfile.j2 +++ b/kolla/docker/nova/nova-libvirt/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='nova', groups='qemu') }} {% if base_package_type == 'rpm' %} @@ -98,3 +100,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start && chmod 744 /usr/local/bin/koll {% block nova_libvirt_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/nova/nova-novncproxy/Dockerfile.j2 b/kolla/docker/nova/nova-novncproxy/Dockerfile.j2 index 4952ed734..2c13db694 100644 --- a/kolla/docker/nova/nova-novncproxy/Dockerfile.j2 +++ b/kolla/docker/nova/nova-novncproxy/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set nova_novncproxy_packages = [ 'novnc' ] %} @@ -17,5 +19,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block nova_novncproxy_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER nova diff --git a/kolla/docker/nova/nova-spicehtml5proxy/Dockerfile.j2 b/kolla/docker/nova/nova-spicehtml5proxy/Dockerfile.j2 index f3a90b0c5..4d22973f5 100644 --- a/kolla/docker/nova/nova-spicehtml5proxy/Dockerfile.j2 +++ b/kolla/docker/nova/nova-spicehtml5proxy/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set nova_spicehtml5proxy_packages = [ @@ -29,5 +31,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block nova_spicehtml5proxy_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER nova diff --git a/kolla/docker/nova/nova-ssh/Dockerfile.j2 b/kolla/docker/nova/nova-ssh/Dockerfile.j2 index 0d9e36b65..dfb269180 100644 --- a/kolla/docker/nova/nova-ssh/Dockerfile.j2 +++ b/kolla/docker/nova/nova-ssh/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='nova', shell='/bin/bash') }} {% set nova_ssh_packages = [ @@ -32,3 +34,4 @@ RUN sed -ri 's/session(\s+)required(\s+)pam_loginuid.so/session\1optional\2pam_l {% block nova_ssh_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/octavia/octavia-api/Dockerfile.j2 b/kolla/docker/octavia/octavia-api/Dockerfile.j2 index ccb7bad11..c5c7406a8 100644 --- a/kolla/docker/octavia/octavia-api/Dockerfile.j2 +++ b/kolla/docker/octavia/octavia-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set octavia_api_packages = [ ] %} @@ -29,3 +31,4 @@ RUN if [ "$(ls /plugins)" ]; then \ {% block octavia_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/octavia/octavia-base/Dockerfile.j2 b/kolla/docker/octavia/octavia-base/Dockerfile.j2 index 81a589a09..9cd040578 100644 --- a/kolla/docker/octavia/octavia-base/Dockerfile.j2 +++ b/kolla/docker/octavia/octavia-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='octavia') }} {% if base_package_type == 'rpm' %} @@ -46,3 +48,4 @@ RUN ln -s /octavia-base-source/* octavia \ {{ macros.kolla_patch_sources() }} {% block octavia_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/octavia/octavia-driver-agent/Dockerfile.j2 b/kolla/docker/octavia/octavia-driver-agent/Dockerfile.j2 index b3519922a..fdd078ed7 100644 --- a/kolla/docker/octavia/octavia-driver-agent/Dockerfile.j2 +++ b/kolla/docker/octavia/octavia-driver-agent/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set octavia_driver_agent_packages = [ ] %} @@ -25,5 +27,6 @@ RUN if [ "$(ls /plugins)" ]; then \ {% block octavia_driver_agent_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER octavia diff --git a/kolla/docker/opensearch/opensearch-dashboards/Dockerfile.j2 b/kolla/docker/opensearch/opensearch-dashboards/Dockerfile.j2 index 3249b62d2..f6bd31587 100644 --- a/kolla/docker/opensearch/opensearch-dashboards/Dockerfile.j2 +++ b/kolla/docker/opensearch/opensearch-dashboards/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='opensearch-dashboards', shell='/bin/bash', homedir='/usr/share/opensearch-dashboards') }} {{ macros.enable_extra_repos(['opensearch-dashboards']) }} @@ -22,5 +24,6 @@ RUN chmod 755 /usr/local/bin/kolla_extend_start {% block opensearch_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER opensearch-dashboards diff --git a/kolla/docker/opensearch/opensearch/Dockerfile.j2 b/kolla/docker/opensearch/opensearch/Dockerfile.j2 index 6ad8fb29a..2765d9985 100644 --- a/kolla/docker/opensearch/opensearch/Dockerfile.j2 +++ b/kolla/docker/opensearch/opensearch/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='opensearch', shell='/bin/bash', homedir='/usr/share/opensearch') }} {{ macros.enable_extra_repos(['opensearch']) }} @@ -28,5 +30,6 @@ RUN chmod 755 /usr/local/bin/kolla_extend_start {% block opensearch_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER opensearch diff --git a/kolla/docker/openstack-base/Dockerfile.j2 b/kolla/docker/openstack-base/Dockerfile.j2 index b8ed2e465..e062045c9 100644 --- a/kolla/docker/openstack-base/Dockerfile.j2 +++ b/kolla/docker/openstack-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {{ macros.enable_extra_repos(['crb', 'epel']) }} @@ -211,3 +213,4 @@ RUN {{ macros.install_pip(['pip', 'wheel', 'setuptools']) }} \ {{ macros.kolla_patch_sources() }} {% block openstack_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/openvswitch/openvswitch-base/Dockerfile.j2 b/kolla/docker/openvswitch/openvswitch-base/Dockerfile.j2 index 44d8df41c..46fe0be83 100644 --- a/kolla/docker/openvswitch/openvswitch-base/Dockerfile.j2 +++ b/kolla/docker/openvswitch/openvswitch-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% block ovs_install %} {% if base_package_type == 'rpm' %} @@ -43,3 +45,4 @@ RUN if [ -e /usr/sbin/tcpdump ]; then mv /usr/sbin/tcpdump /usr/bin/tcpdump; fi {{ macros.kolla_patch_sources() }} {% block openvswitch_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ovn/ovn-base/Dockerfile.j2 b/kolla/docker/ovn/ovn-base/Dockerfile.j2 index 3efd001a3..96893782a 100644 --- a/kolla/docker/ovn/ovn-base/Dockerfile.j2 +++ b/kolla/docker/ovn/ovn-base/Dockerfile.j2 @@ -7,7 +7,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} + +{{ macros.enable_extra_repos(['openvswitch']) }} + {% set ovn_base_packages = [ 'ovn${KOLLA_RPM_OVN_VERSION}', ] %} @@ -27,3 +32,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start {{ macros.kolla_patch_sources() }} {% block ovn_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ovn/ovn-controller/Dockerfile.j2 b/kolla/docker/ovn/ovn-controller/Dockerfile.j2 index f95e61fee..b70db50e6 100644 --- a/kolla/docker/ovn/ovn-controller/Dockerfile.j2 +++ b/kolla/docker/ovn/ovn-controller/Dockerfile.j2 @@ -7,7 +7,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} + +{{ macros.enable_extra_repos(['openvswitch']) }} + {% set ovn_controller_packages = [ 'ovn${KOLLA_RPM_OVN_VERSION}-host' ] %} @@ -27,3 +32,4 @@ RUN chmod 644 /usr/local/bin/kolla_ovn_extend_start {% block ovn_controller_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ovn/ovn-nb-db-server/Dockerfile.j2 b/kolla/docker/ovn/ovn-nb-db-server/Dockerfile.j2 index ea6a53249..472c6e8c9 100644 --- a/kolla/docker/ovn/ovn-nb-db-server/Dockerfile.j2 +++ b/kolla/docker/ovn/ovn-nb-db-server/Dockerfile.j2 @@ -7,7 +7,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} + +{{ macros.enable_extra_repos(['openvswitch']) }} + {% set ovn_nb_db_server_packages = [ 'ovn${KOLLA_RPM_OVN_VERSION}-central', ] %} @@ -29,3 +34,4 @@ RUN chmod 644 /usr/local/bin/kolla_ovn_extend_start {% block ovn_nb_db_server_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ovn/ovn-northd/Dockerfile.j2 b/kolla/docker/ovn/ovn-northd/Dockerfile.j2 index 4a15e10eb..6a9c1b03f 100644 --- a/kolla/docker/ovn/ovn-northd/Dockerfile.j2 +++ b/kolla/docker/ovn/ovn-northd/Dockerfile.j2 @@ -7,7 +7,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} + +{{ macros.enable_extra_repos(['openvswitch']) }} + {% set ovn_northd_packages = [ 'ovn${KOLLA_RPM_OVN_VERSION}-central', ] %} @@ -27,3 +32,4 @@ RUN chmod 644 /usr/local/bin/kolla_ovn_extend_start {% block ovn_northd_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ovn/ovn-sb-db-relay/Dockerfile.j2 b/kolla/docker/ovn/ovn-sb-db-relay/Dockerfile.j2 index 427b6f7f1..1d1761033 100644 --- a/kolla/docker/ovn/ovn-sb-db-relay/Dockerfile.j2 +++ b/kolla/docker/ovn/ovn-sb-db-relay/Dockerfile.j2 @@ -5,6 +5,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block ovn_sb_db_server_header %}{% endblock %} +{% import "macros.j2" as macros with context %} + {% block ovn_sb_db_relay_ovn_ctl %} {# TODO(mnasiadka): Switch to 25.03 branch when available, ideally rpm/deb packages will be available. #} @@ -14,5 +16,6 @@ RUN curl -o /usr/share/ovn/scripts/ovn-ctl https://raw.githubusercontent.com/ovn COPY extend_start.sh /usr/local/bin/kolla_ovn_extend_start RUN chmod 644 /usr/local/bin/kolla_ovn_extend_start +{{ macros.kolla_patch_sources() }} {% block ovn_sb_db_relay_footer %}{% endblock %} {% block footer %}{% endblock %} diff --git a/kolla/docker/ovn/ovn-sb-db-server/Dockerfile.j2 b/kolla/docker/ovn/ovn-sb-db-server/Dockerfile.j2 index afceb8173..419622389 100644 --- a/kolla/docker/ovn/ovn-sb-db-server/Dockerfile.j2 +++ b/kolla/docker/ovn/ovn-sb-db-server/Dockerfile.j2 @@ -7,7 +7,12 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} + +{{ macros.enable_extra_repos(['openvswitch']) }} + {% set ovn_sb_db_server_packages = [ 'ovn${KOLLA_RPM_OVN_VERSION}-central', ] %} @@ -29,3 +34,4 @@ RUN chmod 644 /usr/local/bin/kolla_ovn_extend_start {% block ovn_sb_db_server_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/ovsdpdk/ovsdpdk/Dockerfile.j2 b/kolla/docker/ovsdpdk/ovsdpdk/Dockerfile.j2 index eab4f5474..fc0b55646 100644 --- a/kolla/docker/ovsdpdk/ovsdpdk/Dockerfile.j2 +++ b/kolla/docker/ovsdpdk/ovsdpdk/Dockerfile.j2 @@ -4,6 +4,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% endblock %} {% import "macros.j2" as macros with context %} + +{{ macros.kolla_enable_base_repos() }} {% block ovsdpdk_header %}{% endblock %} {% block ovsdpdk_install %} @@ -11,7 +13,7 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% if base_package_type == 'rpm' %} #TODO(sean-k-mooney) create repo via open suse build system to package -# for centos distro. +# for rocky distro. RUN echo 'image not yet available for {{ base_distro }}' \ && /bin/false @@ -38,3 +40,4 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start {{ macros.kolla_patch_sources() }} {% block ovsdpdk_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/placement/placement-api/Dockerfile.j2 b/kolla/docker/placement/placement-api/Dockerfile.j2 index a13f65bdf..6f5281ca4 100644 --- a/kolla/docker/placement/placement-api/Dockerfile.j2 +++ b/kolla/docker/placement/placement-api/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set placement_api_packages = [ ] %} {{ macros.install_packages(placement_api_packages | customizable("packages")) }} @@ -17,3 +19,4 @@ COPY extend_start.sh /usr/local/bin/kolla_placement_extend_start {% block placement_api_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/placement/placement-base/Dockerfile.j2 b/kolla/docker/placement/placement-base/Dockerfile.j2 index 6454da748..258b09cee 100644 --- a/kolla/docker/placement/placement-base/Dockerfile.j2 +++ b/kolla/docker/placement/placement-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='placement') }} {% if base_package_type == 'rpm' %} @@ -42,3 +44,4 @@ RUN ln -s placement-base-source/* placement \ {{ macros.kolla_patch_sources() }} {% block placement_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/prometheus/prometheus-blackbox-exporter/Dockerfile.j2 b/kolla/docker/prometheus/prometheus-blackbox-exporter/Dockerfile.j2 index 95018facf..f918d4f12 100644 --- a/kolla/docker/prometheus/prometheus-blackbox-exporter/Dockerfile.j2 +++ b/kolla/docker/prometheus/prometheus-blackbox-exporter/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {% set blackbox_packages = [ 'libcap' @@ -32,5 +34,6 @@ RUN ln -s /prometheus-blackbox-exporter-source/* prometheus-blackbox-exporter \ {% block prometheus_blackbox_exporter_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER prometheus diff --git a/kolla/docker/prometheus/prometheus-libvirt-exporter/Dockerfile.j2 b/kolla/docker/prometheus/prometheus-libvirt-exporter/Dockerfile.j2 index c32b2f8c7..cdf01797f 100644 --- a/kolla/docker/prometheus/prometheus-libvirt-exporter/Dockerfile.j2 +++ b/kolla/docker/prometheus/prometheus-libvirt-exporter/Dockerfile.j2 @@ -5,6 +5,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% block prometheus_libvirt_exporter_header %}{% endblock %} {{ macros.install_packages(prometheus_libvirt_exporter_packages | customizable("packages")) }} @@ -21,5 +23,6 @@ RUN ln -s /prometheus-libvirt-exporter-source/* prometheus-libvirt-exporter \ {% block prometheus_libvirt_exporter_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER root diff --git a/kolla/docker/prometheus/prometheus-valkey-exporter/Dockerfile.j2 b/kolla/docker/prometheus/prometheus-valkey-exporter/Dockerfile.j2 index c6cf79ac3..348c7fe2a 100644 --- a/kolla/docker/prometheus/prometheus-valkey-exporter/Dockerfile.j2 +++ b/kolla/docker/prometheus/prometheus-valkey-exporter/Dockerfile.j2 @@ -5,6 +5,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% block prometheus_valkey_exporter_header %}{% endblock %} {{ macros.install_packages(prometheus_valkey_exporter_packages | customizable("packages")) }} @@ -21,5 +23,6 @@ RUN ln -s /prometheus-valkey-exporter-source/* prometheus-valkey-exporter \ {% block prometheus_valkey_exporter_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER prometheus diff --git a/kolla/docker/proxysql/Dockerfile.j2 b/kolla/docker/proxysql/Dockerfile.j2 index a76dbcda8..4ddf7d5cb 100644 --- a/kolla/docker/proxysql/Dockerfile.j2 +++ b/kolla/docker/proxysql/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.enable_extra_repos(['proxysql']) }} {% set proxysql_packages = [ @@ -39,5 +41,6 @@ RUN install -g proxysql -o proxysql -m 770 -d /var/lib/kolla/proxysql \ {% block proxysql_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER proxysql diff --git a/kolla/docker/rabbitmq/Dockerfile.j2 b/kolla/docker/rabbitmq/Dockerfile.j2 index bd600785b..79c762e4c 100644 --- a/kolla/docker/rabbitmq/Dockerfile.j2 +++ b/kolla/docker/rabbitmq/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='rabbitmq') }} {{ macros.enable_extra_repos(['erlang', 'rabbitmq']) }} @@ -65,5 +67,6 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start {% block rabbitmq_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER rabbitmq diff --git a/kolla/docker/skyline/skyline-apiserver/Dockerfile.j2 b/kolla/docker/skyline/skyline-apiserver/Dockerfile.j2 index 85b40d51c..e50b0467d 100644 --- a/kolla/docker/skyline/skyline-apiserver/Dockerfile.j2 +++ b/kolla/docker/skyline/skyline-apiserver/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + ADD skyline-apiserver-archive /skyline-apiserver-source {% set skyline_apiserver_pip_packages = [ @@ -23,5 +25,6 @@ RUN ln -s skyline-apiserver-source/* skyline-apiserver \ {% block skyline_apiserver_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER skyline diff --git a/kolla/docker/skyline/skyline-console/Dockerfile.j2 b/kolla/docker/skyline/skyline-console/Dockerfile.j2 index ba62f5041..9b34f503f 100644 --- a/kolla/docker/skyline/skyline-console/Dockerfile.j2 +++ b/kolla/docker/skyline/skyline-console/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% set skyline_console_packages = [ 'nginx' ] %} @@ -26,5 +28,6 @@ RUN ln -s skyline-console-source/* skyline-console \ {% block skyline_console_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER root diff --git a/kolla/docker/tacker/tacker-base/Dockerfile.j2 b/kolla/docker/tacker/tacker-base/Dockerfile.j2 index c24c8d87c..e84a4fe21 100644 --- a/kolla/docker/tacker/tacker-base/Dockerfile.j2 +++ b/kolla/docker/tacker/tacker-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='tacker') }} {% if base_package_type == 'rpm' %} @@ -41,3 +43,4 @@ RUN ln -s tacker-base-source/* tacker \ {{ macros.kolla_patch_sources() }} {% block tacker_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/tgtd/Dockerfile.j2 b/kolla/docker/tgtd/Dockerfile.j2 index b8cbbaaf5..864aa1232 100644 --- a/kolla/docker/tgtd/Dockerfile.j2 +++ b/kolla/docker/tgtd/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} RUN echo 'Not available for {{ base_distro }}' \ @@ -24,3 +26,4 @@ RUN sed -i '1 i include /var/lib/cinder/volumes/*' /etc/tgt/targets.conf {% block tgtd_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/trove/trove-base/Dockerfile.j2 b/kolla/docker/trove/trove-base/Dockerfile.j2 index a675b2b72..5309b744e 100644 --- a/kolla/docker/trove/trove-base/Dockerfile.j2 +++ b/kolla/docker/trove/trove-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='trove') }} ADD trove-base-archive /trove-base-source @@ -28,3 +30,4 @@ RUN ln -s trove-base-source/* trove \ {{ macros.kolla_patch_sources() }} {% block trove_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/docker/valkey/valkey-sentinel/Dockerfile.j2 b/kolla/docker/valkey/valkey-sentinel/Dockerfile.j2 index 94828a38f..e9b89113b 100644 --- a/kolla/docker/valkey/valkey-sentinel/Dockerfile.j2 +++ b/kolla/docker/valkey/valkey-sentinel/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {{ macros.enable_extra_repos(['epel']) }} {% set valkey_sentinel_packages = ['valkey'] %} @@ -22,5 +24,6 @@ RUN chmod 644 /usr/local/bin/kolla_extend_start {% block valkey_sentinel_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER valkey diff --git a/kolla/docker/valkey/valkey-server/Dockerfile.j2 b/kolla/docker/valkey/valkey-server/Dockerfile.j2 index be157d89e..e0005a9e0 100644 --- a/kolla/docker/valkey/valkey-server/Dockerfile.j2 +++ b/kolla/docker/valkey/valkey-server/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {% if base_package_type == 'rpm' %} {{ macros.enable_extra_repos(['epel']) }} {% endif %} @@ -19,5 +21,6 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% block valkey_server_footer %}{% endblock %} {% block footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} USER valkey diff --git a/kolla/docker/watcher/watcher-base/Dockerfile.j2 b/kolla/docker/watcher/watcher-base/Dockerfile.j2 index 8de6ceb9d..e5530842d 100644 --- a/kolla/docker/watcher/watcher-base/Dockerfile.j2 +++ b/kolla/docker/watcher/watcher-base/Dockerfile.j2 @@ -7,6 +7,8 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" {% import "macros.j2" as macros with context %} +{{ macros.kolla_enable_base_repos() }} + {{ macros.configure_user(name='watcher') }} ADD watcher-base-archive /watcher-base-source @@ -27,3 +29,4 @@ RUN ln -s watcher-base-source/* watcher \ {{ macros.kolla_patch_sources() }} {% block watcher_base_footer %}{% endblock %} +{{ macros.kolla_cleanup_repos() }} diff --git a/kolla/image/kolla_worker.py b/kolla/image/kolla_worker.py index 36910e10a..837bd797c 100644 --- a/kolla/image/kolla_worker.py +++ b/kolla/image/kolla_worker.py @@ -120,7 +120,7 @@ class KollaWorker(object): conf.rpm_setup_config if repo_file is not None]) self.rpm_setup = self.build_rpm_setup(rpm_setup_config) - if self.base in ['centos', 'rocky']: + if self.base == 'rocky': self.distro_package_manager = 'dnf' self.base_package_type = 'rpm' elif self.base in ['debian']: @@ -329,6 +329,7 @@ class KollaWorker(object): return { 'debian_package_install': jinja_methods.debian_package_install, 'handle_repos': jinja_methods.handle_repos, + 'cleanup_repos': jinja_methods.cleanup_repos, 'raise_error': jinja_methods.raise_error, } diff --git a/kolla/image/tasks.py b/kolla/image/tasks.py index ba2ade743..87dff5640 100644 --- a/kolla/image/tasks.py +++ b/kolla/image/tasks.py @@ -56,6 +56,12 @@ def normalize_tarinfo(tarinfo): # unchanged. An integer mtime also stops tarfile's PAX writer # emitting per-entry float-mtime extended headers. tarinfo.mtime = 0 + + # Ensure directories are writable. Git repos contain directories + # such as .git/objects/pack/ with mode 0555, which prevents + # non-root users from re-extracting into an existing items_path + # on build retry. + tarinfo.mode |= 0o200 return tarinfo diff --git a/kolla/image/unbuildable.py b/kolla/image/unbuildable.py index 5dbc72094..6d7f4eb5c 100644 --- a/kolla/image/unbuildable.py +++ b/kolla/image/unbuildable.py @@ -19,16 +19,6 @@ UNBUILDABLE_IMAGES = { "bifrost-base", # someone need to get upstream working first }, - # Issues for SHA1 keys: - # https://github.com/grafana/grafana/issues/41036 - 'centos': { - "collectd", # CS10 no opstools repo - "hacluster-pcs", # Missing crmsh package - "nova-spicehtml5proxy", # Missing spicehtml5 package - "ovsdpdk", # Not supported on CentOS - "tgtd", # Not supported on CentOS - }, - 'debian': { }, @@ -36,8 +26,8 @@ UNBUILDABLE_IMAGES = { "collectd", # Missing libpqos.so.4()(64bit) "hacluster-pcs", # Missing crmsh package "nova-spicehtml5proxy", # Missing spicehtml5 package - "ovsdpdk", # Not supported on CentOS - "tgtd", # Not supported on CentOS + "ovsdpdk", # Not supported on Rocky + "tgtd", # Not supported on Rocky }, 'ubuntu': { @@ -45,7 +35,4 @@ UNBUILDABLE_IMAGES = { 'ubuntu+aarch64': { }, - - 'centos+aarch64': { - }, } diff --git a/kolla/template/methods.py b/kolla/template/methods.py index 8bb7a7fc3..461003fa0 100644 --- a/kolla/template/methods.py +++ b/kolla/template/methods.py @@ -45,6 +45,22 @@ echo 'enabled=1' >>/etc/yum.repos.d/{repo}.repo" DNF_REPO_GPGCHECK = " && echo 'repo_gpgcheck={repo_gpgcheck}' \ >>/etc/yum.repos.d/{repo}.repo" +BACKUP_DIR = '/tmp/kolla-repos-backup' # nosec B108 +APT_BACKUP = ( + 'mkdir -p {backup_dir}' + ' && cp /etc/apt/sources.list.d/{repo}.sources' + ' {backup_dir}/{repo}.sources 2>/dev/null || true' + ' && touch {backup_dir}/{repo}.enabled' + ' && ' +) +RPM_BACKUP = ( + 'mkdir -p {backup_dir}' + ' && cp /etc/yum.repos.d/{file}' + ' {backup_dir}/{file} 2>/dev/null || true' + ' && touch {backup_dir}/{file}.enabled' + ' && ' +) + def debian_package_install(packages, clean_package_cache=True): """Jinja utility method for building debian-based package install command. @@ -99,6 +115,41 @@ def debian_package_install(packages, clean_package_cache=True): return ' && '.join(cmds) +def _load_repos(repos_yaml_override=None): + """Load default repos.yaml and merge an optional override file.""" + default_repofile = os.path.dirname( + os.path.realpath(__file__)) + '/repos.yaml' + with open(default_repofile, 'r') as f: + repo_data = yaml.safe_load(f) + + if repos_yaml_override: + with open(repos_yaml_override, 'r') as f: + for section, repos in yaml.safe_load(f).items(): + if section in repo_data: + repo_data[section].update(repos) + else: + repo_data[section] = repos + + return repo_data + + +def _build_repo_list(repo_data, base_package_type, base_distro, base_arch): + """Flatten repo_data sections for the current distro/arch into one dict.""" + result = {} + for section in (base_package_type, base_distro, + '%s-%s' % (base_distro, base_arch)): + for repo_name, repo_info in repo_data.get(section, {}).items(): + if repo_name in result: + merged = {**result[repo_name], **repo_info} + if any(k in merged for k in + ('baseurl', 'metalink', 'mirrorlist', 'url')): + merged.pop('distro', None) + result[repo_name] = merged + else: + result[repo_name] = repo_info + return result + + @pass_context def handle_repos(context, reponames, mode): """Generate Dockerfile RUN commands to enable or disable package repos. @@ -110,29 +161,19 @@ def handle_repos(context, reponames, mode): Repo names are distro/arch-agnostic aliases that map to the real repo name and URL for the target platform. So 'ceph' becomes - 'centos-ceph-squid' on CentOS, uses UCA on Ubuntu, and so on. + 'centos-ceph-squid' on Rocky, uses UCA on Ubuntu, and so on. Distro/arch sections are not required to have all entries - missing ones are silently ignored. - NOTE(hrw): we need to handle CentOS, Debian and Ubuntu with one macro. + NOTE(hrw): we need to handle Debian, Rocky and Ubuntu with one macro. """ if not isinstance(reponames, list): raise TypeError("First argument should be a list of repositories") - default_repofile = os.path.dirname( - os.path.realpath(__file__)) + '/repos.yaml' - with open(default_repofile, 'r') as repos_file: - repo_data = yaml.safe_load(repos_file) - default_repo_data = {k: dict(v) for k, v in repo_data.items()} - - if context.get('repos_yaml'): - with open(context.get('repos_yaml'), 'r') as repos_file: - for section, repos in yaml.safe_load(repos_file).items(): - if section in repo_data: - repo_data[section].update(repos) - else: - repo_data[section] = repos + repos_yaml = context.get('repos_yaml') + repo_data = _load_repos(repos_yaml) + default_repo_data = _load_repos() base_package_type = context.get('base_package_type') base_distro = context.get('base_distro') @@ -141,30 +182,23 @@ def handle_repos(context, reponames, mode): openstack_release_codename = context.get('openstack_release_codename') commands = '' - - def _build_repo_list(data): - result = {} - for section in (base_package_type, base_distro, - '%s-%s' % (base_distro, base_arch)): - for repo_name, repo_info in data.get(section, {}).items(): - if repo_name in result: - merged = {**result[repo_name], **repo_info} - if any(k in merged for k in - ('baseurl', 'metalink', 'mirrorlist', 'url')): - merged.pop('distro', None) - result[repo_name] = merged - else: - result[repo_name] = repo_info - return result + backed_up_files = set() try: - repo_list = _build_repo_list(repo_data) + repo_list = _build_repo_list( + repo_data, base_package_type, base_distro, base_arch) except KeyError: # NOTE(hrw): Fallback to distro list repo_list = repo_data[base_distro] - if base_package_type == 'rpm' and context.get('repos_yaml'): - default_repo_list = _build_repo_list(default_repo_data) + # NOTE: repos_yaml overrides replace a repo's whole entry (see + # _load_repos), so an override that doesn't restate 'file_group' would + # otherwise lose it. Keep the default list around to fall back to the + # distro's real on-disk layout for repos sharing a file. + default_repo_list = _build_repo_list( + default_repo_data, base_package_type, base_distro, base_arch) + + if base_package_type == 'rpm' and repos_yaml: distro_overridden = {r for r, d in repo_list.items() if not d.get('distro') and default_repo_list.get(r, {}).get('distro')} @@ -192,6 +226,21 @@ def handle_repos(context, reponames, mode): if base_package_type == 'rpm': if mode == 'enable': if not _repo.get('distro'): + if _repo.get('build_only'): + # NOTE: some distros (e.g. Rocky's baseos, + # appstream and crb) share a single .repo file, + # so back up that shared file, not one named + # after this repo, or the original content is + # lost once handle_repos rewrites the file. + repo_file = ( + _repo.get('file_group') + or default_repo_list.get(repo, {}).get( + 'file_group') + or '{}.repo'.format(repo)) + if repo_file not in backed_up_files: + commands += RPM_BACKUP.format( + backup_dir=BACKUP_DIR, file=repo_file) + backed_up_files.add(repo_file) commands += DNF_REMOVE_EXISTING.format( name=_repo['name']) commands += " && " @@ -252,6 +301,13 @@ def handle_repos(context, reponames, mode): elif base_package_type == "deb": if mode == "enable" and not _repo.get('distro'): + if _repo.get('build_only'): + sources_file = ( + '/etc/apt/sources.list.d/{}.sources'.format(repo)) + if sources_file not in backed_up_files: + commands += APT_BACKUP.format( + backup_dir=BACKUP_DIR, repo=repo) + backed_up_files.add(sources_file) gpg_key = _repo['gpg_key'] signed_by = gpg_key if gpg_key.startswith('/') \ else '/etc/kolla/apt-keys/' + gpg_key @@ -286,5 +342,76 @@ def handle_repos(context, reponames, mode): return commands +def get_cleanup_commands(repos_yaml, base_package_type, base_distro, + base_arch): + """Return a RUN command string restoring or removing build_only repos. + + Repos marked build_only: true are backed up before being overwritten in + handle_repos(). This function generates the matching restore commands so + that mirror URLs are not baked into the final image. Repos that had no + prior file (no backup exists) are deleted instead of restored. + + Returns an empty string when there are no build_only repos. + """ + repo_data = _load_repos(repos_yaml) + repo_list = _build_repo_list( + repo_data, base_package_type, base_distro, base_arch) + default_repo_list = _build_repo_list( + _load_repos(), base_package_type, base_distro, base_arch) + + cleanup_cmds = [] + restored_file_groups = set() + for repo_name, repo_info in repo_list.items(): + if not repo_info.get('build_only'): + continue + if base_package_type == 'rpm' and not repo_info.get('distro'): + own_file = '{}.repo'.format(repo_name) + file_group = ( + repo_info.get('file_group') + or default_repo_list.get(repo_name, {}).get('file_group') + or own_file) + if file_group != own_file: + # handle_repos always writes this repo's override to + # own_file, regardless of where the original repo lived, + # so it always needs dropping on its own. + cleanup_cmds.append( + 'rm -f /etc/yum.repos.d/{}'.format(own_file)) + if file_group in restored_file_groups: + continue + restored_file_groups.add(file_group) + cleanup_cmds.append( + '[ -f {backup_dir}/{file}.enabled ]' + ' && ( mv {backup_dir}/{file}' + ' /etc/yum.repos.d/{file} 2>/dev/null' + ' || rm -f /etc/yum.repos.d/{file} )' + ' || true'.format( + backup_dir=BACKUP_DIR, file=file_group)) + elif base_package_type == 'deb' and not repo_info.get('distro'): + cleanup_cmds.append( + '[ -f {backup_dir}/{repo}.enabled ]' + ' && ( mv {backup_dir}/{repo}.sources' + ' /etc/apt/sources.list.d/{repo}.sources 2>/dev/null' + ' || rm -f /etc/apt/sources.list.d/{repo}.sources )' + ' || true'.format( + backup_dir=BACKUP_DIR, repo=repo_name)) + + if not cleanup_cmds: + return '' + + cleanup_cmds.append('rm -rf {}'.format(BACKUP_DIR)) + return 'RUN ' + ' \\\n && '.join(cleanup_cmds) + + +@pass_context +def cleanup_repos(context): + """Jinja2-callable wrapper around get_cleanup_commands().""" + return get_cleanup_commands( + context.get('repos_yaml'), + context.get('base_package_type'), + context.get('base_distro'), + context.get('base_arch'), + ) + + def raise_error(msg: str) -> t.NoReturn: raise Exception(msg) diff --git a/kolla/template/repos.yaml b/kolla/template/repos.yaml index 42e0185c4..1ae76d137 100644 --- a/kolla/template/repos.yaml +++ b/kolla/template/repos.yaml @@ -195,24 +195,6 @@ rpm: https://github.com/rabbitmq/signing-keys/releases/download/3.0/rabbitmq-release-signing-key.asc name: "rabbitmq_rabbitmq-server" -centos: - appstream: - distro: True - file_group: "centos.repo" - name: "appstream" - baseos: - distro: True - file_group: "centos.repo" - name: "baseos" - crb: - file_group: "centos.repo" - -centos-aarch64: - erlang: - baseurl: "https://download.copr.fedorainfracloud.org/results/@openstack-kolla/rabbitmq-erlang-27/rhel-$releasever-aarch64/" - gpgkey: "https://download.copr.fedorainfracloud.org/results/@openstack-kolla/rabbitmq-erlang-27/pubkey.gpg" - name: "copr-rabbitmq-erlang" - rocky: appstream: distro: True diff --git a/kolla/tests/test_build.py b/kolla/tests/test_build.py index 48023cc1c..7d7c58d0f 100644 --- a/kolla/tests/test_build.py +++ b/kolla/tests/test_build.py @@ -820,7 +820,7 @@ class KollaWorkerTest(base.TestCase): self.mock_client = patcher.start() def test_supported_base_distro(self): - build_base = ['centos', 'debian', 'ubuntu'] + build_base = ['debian', 'rocky', 'ubuntu'] for base_distro in build_base: self.conf.set_override('base', base_distro) diff --git a/kolla/tests/test_methods.py b/kolla/tests/test_methods.py index 8a577aead..9b5dd212a 100644 --- a/kolla/tests/test_methods.py +++ b/kolla/tests/test_methods.py @@ -10,6 +10,7 @@ # See the License for the specific language governing permissions and # limitations under the License. +import os import tempfile import yaml @@ -26,10 +27,10 @@ class MethodsTest(base.TestCase): expectCmd = 'apt-get -y install --no-install-recommends package2.deb' self.assertEqual(expectCmd, result.split("&&")[1].strip()) - def test_enable_repos_centos_baseurl(self): + def test_enable_repos_rocky_baseurl(self): template_vars = { 'base_arch': 'x86_64', - 'base_distro': 'centos', + 'base_distro': 'rocky', 'base_package_type': 'rpm', } @@ -48,10 +49,10 @@ class MethodsTest(base.TestCase): expectCmd += ">>/etc/yum.repos.d/grafana.repo" self.assertEqual(expectCmd, result) - def test_enable_repos_centos_ceph_distro(self): + def test_enable_repos_rocky_ceph_distro(self): template_vars = { 'base_arch': 'x86_64', - 'base_distro': 'centos', + 'base_distro': 'rocky', 'base_package_type': 'rpm', } @@ -59,10 +60,10 @@ class MethodsTest(base.TestCase): expectCmd = "RUN dnf config-manager --enable centos-ceph-squid || true" self.assertEqual(expectCmd, result) - def test_enable_repos_centos_arch(self): + def test_enable_repos_rocky_arch(self): template_vars = { "base_arch": "aarch64", - "base_distro": "centos", + "base_distro": "rocky", "base_package_type": "rpm", } @@ -81,10 +82,10 @@ class MethodsTest(base.TestCase): expectCmd += ">>/etc/yum.repos.d/grafana.repo" self.assertEqual(expectCmd, result) - def test_enable_repos_centos_multiple(self): + def test_enable_repos_rocky_multiple(self): template_vars = { "base_arch": "x86_64", - "base_distro": "centos", + "base_distro": "rocky", "base_package_type": "rpm", } @@ -121,10 +122,10 @@ class MethodsTest(base.TestCase): expectCmd += ">>/etc/yum.repos.d/rabbitmq.repo" self.assertEqual(expectCmd, result) - def test_enable_repos_centos_distro_enable(self): + def test_enable_repos_rocky_distro_enable(self): template_vars = { "base_arch": "x86_64", - "base_distro": "centos", + "base_distro": "rocky", "base_package_type": "rpm", } @@ -133,10 +134,10 @@ class MethodsTest(base.TestCase): expectCmd = "RUN dnf config-manager --enable crb || true" self.assertEqual(expectCmd, result) - def test_enable_repos_centos_distro_disable(self): + def test_enable_repos_rocky_distro_disable(self): template_vars = { "base_arch": "x86_64", - "base_distro": "centos", + "base_distro": "rocky", "base_package_type": "rpm", } @@ -144,10 +145,10 @@ class MethodsTest(base.TestCase): expectCmd = "RUN dnf config-manager --disable crb || true" self.assertEqual(expectCmd, result) - def test_enable_repos_centos_distro_enable_multiple(self): + def test_enable_repos_rocky_distro_enable_multiple(self): template_vars = { "base_arch": "x86_64", - "base_distro": "centos", + "base_distro": "rocky", "base_package_type": "rpm", } @@ -315,17 +316,28 @@ class MethodsTest(base.TestCase): 'Signed-By: /usr/share/keyrings/test.gpg', result) def test_repos_yaml_merge_keeps_default_repos(self): - repos = {'rocky': {'baseos': { - 'name': 'baseos', - 'baseurl': 'http://mirror.example.com/rocky/10/BaseOS/$basearch/', - 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10', - }}} + mirror_url = ("http://mirror.example.com/rocky/10/{component}/" + "$basearch/") + repos = {'rocky': { + 'appstream': { + 'name': 'appstream', + 'baseurl': mirror_url.format(component='AppStream'), + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10'}, + 'baseos': { + 'name': 'baseos', + 'baseurl': mirror_url.format(component='BaseOS'), + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10'}, + 'crb': { + 'name': 'crb', + 'baseurl': mirror_url.format(component='CRB'), + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10'}, + }} with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml') as f: yaml.dump(repos, f) f.flush() template_vars = { 'base_arch': 'x86_64', - 'base_distro': 'centos', + 'base_distro': 'rocky', 'base_package_type': 'rpm', 'repos_yaml': f.name, } @@ -374,7 +386,7 @@ class MethodsTest(base.TestCase): f.flush() template_vars = { 'base_arch': 'x86_64', - 'base_distro': 'centos', + 'base_distro': 'rocky', 'base_package_type': 'rpm', 'repos_yaml': f.name, } @@ -391,13 +403,385 @@ class MethodsTest(base.TestCase): f.flush() template_vars = { 'base_arch': 'x86_64', - 'base_distro': 'centos', + 'base_distro': 'rocky', 'base_package_type': 'rpm', 'repos_yaml': f.name, } self.assertRaises(ValueError, methods.handle_repos, template_vars, ['test-repo'], 'enable') + def test_handle_repos_deb_build_only_backup(self): + repos = {'debian': {'build-only-mirror': { + 'build_only': True, + 'url': 'http://mirror.example.com/debian', + 'suite': 'trixie', + 'component': 'main', + 'gpg_key': '/usr/share/keyrings/debian-archive-keyring.gpg', + }}} + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(repos, f) + fname = f.name + try: + template_vars = { + 'base_arch': 'x86_64', + 'base_distro': 'debian', + 'base_package_type': 'deb', + 'openstack_release_codename': 'Gazpacho', + 'repos_yaml': fname, + } + result = methods.handle_repos( + template_vars, ['build-only-mirror'], 'enable') + self.assertIn('mkdir -p /tmp/kolla-repos-backup', result) + self.assertIn( + 'cp /etc/apt/sources.list.d/build-only-mirror.sources' + ' /tmp/kolla-repos-backup/build-only-mirror.sources', + result) + self.assertIn( + 'touch /tmp/kolla-repos-backup/build-only-mirror.enabled', + result) + # backup must precede the actual repo write + self.assertLess( + result.index('cp /etc/apt/sources.list.d/build-only-mirror'), + result.index("echo 'Uris:")) + finally: + os.unlink(fname) + + def test_handle_repos_rpm_build_only_backup(self): + repos = {'rpm': {'build-only-mirror': { + 'build_only': True, + 'name': 'build-only-mirror', + 'baseurl': 'http://mirror.example.com/rpm/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10', + }}} + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(repos, f) + fname = f.name + try: + template_vars = { + 'base_arch': 'x86_64', + 'base_distro': 'centos', + 'base_package_type': 'rpm', + 'repos_yaml': fname, + } + result = methods.handle_repos( + template_vars, ['build-only-mirror'], 'enable') + self.assertIn('mkdir -p /tmp/kolla-repos-backup', result) + self.assertIn( + 'cp /etc/yum.repos.d/build-only-mirror.repo' + ' /tmp/kolla-repos-backup/build-only-mirror.repo', + result) + self.assertIn( + 'touch /tmp/kolla-repos-backup/build-only-mirror.repo' + '.enabled', + result) + # backup must precede DNF_REMOVE_EXISTING + self.assertLess( + result.index( + 'cp /etc/yum.repos.d/build-only-mirror.repo'), + result.index('grep -rlF')) + finally: + os.unlink(fname) + + def test_handle_repos_rpm_shared_file_group_backup(self): + """Repos sharing a file_group back up that file once, by its name. + + e.g. Rocky's baseos/appstream/crb all live in rocky.repo. + """ + repos = {'rpm': { + 'shared-a': { + 'build_only': True, + 'file_group': 'shared.repo', + 'name': 'shared-a', + 'baseurl': 'http://mirror.example.com/a/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-a', + }, + 'shared-b': { + 'build_only': True, + 'file_group': 'shared.repo', + 'name': 'shared-b', + 'baseurl': 'http://mirror.example.com/b/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-b', + }, + }} + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(repos, f) + fname = f.name + try: + template_vars = { + 'base_arch': 'x86_64', + 'base_distro': 'centos', + 'base_package_type': 'rpm', + 'repos_yaml': fname, + } + result = methods.handle_repos( + template_vars, ['shared-a', 'shared-b'], 'enable') + self.assertEqual( + 1, + result.count( + 'cp /etc/yum.repos.d/shared.repo' + ' /tmp/kolla-repos-backup/shared.repo')) + self.assertNotIn('shared-a.repo /tmp/kolla-repos-backup', + result) + self.assertNotIn('shared-b.repo /tmp/kolla-repos-backup', + result) + # each repo still gets its own override file written + self.assertIn('>/etc/yum.repos.d/shared-a.repo', result) + self.assertIn('>/etc/yum.repos.d/shared-b.repo', result) + finally: + os.unlink(fname) + + def test_get_cleanup_commands_rpm_shared_file_group(self): + """The shared file is restored once, each own override always.""" + repos = {'rpm': { + 'shared-a': { + 'build_only': True, + 'file_group': 'shared.repo', + 'name': 'shared-a', + 'baseurl': 'http://mirror.example.com/a/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-a', + }, + 'shared-b': { + 'build_only': True, + 'file_group': 'shared.repo', + 'name': 'shared-b', + 'baseurl': 'http://mirror.example.com/b/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-b', + }, + }} + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(repos, f) + fname = f.name + try: + result = methods.get_cleanup_commands( + fname, 'rpm', 'centos', 'x86_64') + self.assertEqual( + 1, + result.count( + 'mv /tmp/kolla-repos-backup/shared.repo' + ' /etc/yum.repos.d/shared.repo')) + self.assertIn('rm -f /etc/yum.repos.d/shared-a.repo', result) + self.assertIn('rm -f /etc/yum.repos.d/shared-b.repo', result) + finally: + os.unlink(fname) + + def _rocky_mirror_override(self): + """The exact override shape documented for Rocky, minus file_group. + + repos_yaml overrides replace a repo's whole entry, so this matches + what a real user would write per doc/source/admin/image-building + .rst - it must not restate 'file_group' for the fix to be tested. + """ + return {'rocky': { + 'baseos': { + 'build_only': True, + 'name': 'baseos', + 'baseurl': 'http://mirror.example.com/rocky/10/BaseOS/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10', + }, + 'appstream': { + 'build_only': True, + 'name': 'appstream', + 'baseurl': 'http://mirror.example.com/rocky/10/AppStream/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10', + }, + 'crb': { + 'build_only': True, + 'name': 'crb', + 'baseurl': 'http://mirror.example.com/rocky/10/CRB/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10', + }, + }} + + def test_handle_repos_rocky_baseos_backs_up_shared_file(self): + """file_group must be resolved even if the override omits it. + + Rocky's baseos/appstream/crb live in rocky.repo, not their own + .repo files, and a repos_yaml override doesn't restate file_group. + """ + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(self._rocky_mirror_override(), f) + fname = f.name + try: + template_vars = { + 'base_arch': 'x86_64', + 'base_distro': 'rocky', + 'base_package_type': 'rpm', + 'repos_yaml': fname, + } + result = methods.handle_repos( + template_vars, ['baseos', 'appstream'], 'enable') + self.assertEqual( + 1, + result.count( + 'cp /etc/yum.repos.d/rocky.repo' + ' /tmp/kolla-repos-backup/rocky.repo')) + self.assertNotIn('baseos.repo /tmp/kolla-repos-backup', result) + self.assertNotIn('appstream.repo /tmp/kolla-repos-backup', + result) + finally: + os.unlink(fname) + + def test_get_cleanup_commands_rocky_restores_shared_file(self): + """Restoring baseos/appstream/crb recovers the shared rocky.repo.""" + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(self._rocky_mirror_override(), f) + fname = f.name + try: + result = methods.get_cleanup_commands( + fname, 'rpm', 'rocky', 'x86_64') + self.assertEqual( + 1, + result.count( + 'mv /tmp/kolla-repos-backup/rocky.repo' + ' /etc/yum.repos.d/rocky.repo')) + self.assertIn('rm -f /etc/yum.repos.d/baseos.repo', result) + self.assertIn('rm -f /etc/yum.repos.d/appstream.repo', result) + self.assertIn('rm -f /etc/yum.repos.d/crb.repo', result) + finally: + os.unlink(fname) + + def test_handle_repos_build_only_no_double_backup(self): + """Each distinct file is backed up exactly once per handle_repos.""" + repos = {'debian': { + 'mirror-a': { + 'build_only': True, + 'url': 'http://mirror.example.com/a', + 'suite': 'trixie', + 'component': 'main', + 'gpg_key': '/usr/share/keyrings/debian-archive-keyring.gpg', + }, + 'mirror-b': { + 'build_only': True, + 'url': 'http://mirror.example.com/b', + 'suite': 'trixie', + 'component': 'main', + 'gpg_key': '/usr/share/keyrings/debian-archive-keyring.gpg', + }, + }} + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(repos, f) + fname = f.name + try: + template_vars = { + 'base_arch': 'x86_64', + 'base_distro': 'debian', + 'base_package_type': 'deb', + 'openstack_release_codename': 'Gazpacho', + 'repos_yaml': fname, + } + result = methods.handle_repos( + template_vars, ['mirror-a', 'mirror-b'], 'enable') + self.assertEqual( + 1, + result.count( + 'cp /etc/apt/sources.list.d/mirror-a.sources' + ' /tmp/kolla-repos-backup/mirror-a.sources')) + self.assertEqual( + 1, + result.count( + 'cp /etc/apt/sources.list.d/mirror-b.sources' + ' /tmp/kolla-repos-backup/mirror-b.sources')) + finally: + os.unlink(fname) + + def test_handle_repos_without_build_only_no_backup(self): + template_vars = { + 'base_arch': 'x86_64', + 'base_distro': 'debian', + 'base_package_type': 'deb', + 'openstack_release_codename': 'Gazpacho', + } + result = methods.handle_repos(template_vars, ['grafana'], 'enable') + self.assertNotIn('kolla-repos-backup', result) + + def test_get_cleanup_commands_deb(self): + repos = {'debian': {'build-only-mirror': { + 'build_only': True, + 'url': 'http://mirror.example.com/debian', + 'suite': 'trixie', + 'component': 'main', + 'gpg_key': '/usr/share/keyrings/debian-archive-keyring.gpg', + }}} + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(repos, f) + fname = f.name + try: + result = methods.get_cleanup_commands( + fname, 'deb', 'debian', 'x86_64') + self.assertTrue(result.startswith('RUN ')) + self.assertIn( + '[ -f /tmp/kolla-repos-backup/build-only-mirror.enabled ]', + result) + self.assertIn( + 'mv /tmp/kolla-repos-backup/build-only-mirror.sources' + ' /etc/apt/sources.list.d/build-only-mirror.sources', + result) + self.assertIn( + 'rm -f /etc/apt/sources.list.d/build-only-mirror.sources', + result) + self.assertIn('rm -rf /tmp/kolla-repos-backup', result) + finally: + os.unlink(fname) + + def test_get_cleanup_commands_rpm(self): + repos = {'rpm': {'build-only-mirror': { + 'build_only': True, + 'name': 'build-only-mirror', + 'baseurl': 'http://mirror.example.com/', + 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10', + }}} + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(repos, f) + fname = f.name + try: + result = methods.get_cleanup_commands( + fname, 'rpm', 'centos', 'x86_64') + self.assertTrue(result.startswith('RUN ')) + self.assertIn( + '[ -f /tmp/kolla-repos-backup/build-only-mirror.repo' + '.enabled ]', + result) + self.assertIn( + 'mv /tmp/kolla-repos-backup/build-only-mirror.repo' + ' /etc/yum.repos.d/build-only-mirror.repo', + result) + self.assertIn( + 'rm -f /etc/yum.repos.d/build-only-mirror.repo', + result) + self.assertIn('rm -rf /tmp/kolla-repos-backup', result) + finally: + os.unlink(fname) + + def test_get_cleanup_commands_empty_without_build_only(self): + result = methods.get_cleanup_commands(None, 'deb', 'debian', 'x86_64') + self.assertEqual('', result) + + def test_get_cleanup_commands_skips_distro_repos(self): + """distro: True repos are a no-op in handle_repos.""" + repos = {'debian': {'debian': { + 'build_only': True, + 'distro': True, + }}} + with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', + delete=False) as f: + yaml.dump(repos, f) + fname = f.name + try: + result = methods.get_cleanup_commands( + fname, 'deb', 'debian', 'x86_64') + self.assertEqual('', result) + finally: + os.unlink(fname) + def test_repos_yaml_rpm_section_override_not_blocked_by_distro_section( self): """Overriding distro repos in 'rpm' section must not be undone. @@ -405,7 +789,7 @@ class MethodsTest(base.TestCase): The more-specific distro section must not merge distro:True back on top of a URL-bearing override from the generic 'rpm' section. """ - base = 'http://mirror.example.com/centos/10' + base = 'http://mirror.example.com/rocky/10' gpgkey = 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial' repos = {'rpm': { 'baseos': { @@ -429,7 +813,7 @@ class MethodsTest(base.TestCase): f.flush() template_vars = { 'base_arch': 'x86_64', - 'base_distro': 'centos', + 'base_distro': 'rocky', 'base_package_type': 'rpm', 'repos_yaml': f.name, } diff --git a/lint-requirements.txt b/lint-requirements.txt index 25e3687fa..41f6c792e 100644 --- a/lint-requirements.txt +++ b/lint-requirements.txt @@ -1 +1 @@ -ansible-lint===26.6.0 # MIT +ansible-lint===26.8.0 # MIT diff --git a/releasenotes/notes/bug-2163314-d99dd8d0f7a76a95.yaml b/releasenotes/notes/bug-2163314-d99dd8d0f7a76a95.yaml new file mode 100644 index 000000000..0683c8ced --- /dev/null +++ b/releasenotes/notes/bug-2163314-d99dd8d0f7a76a95.yaml @@ -0,0 +1,6 @@ +--- +fixes: + - | + Fixes child image builds from ``openvswitch-base`` with a custom + OpenvSwitch repo. + `LP#2163314 <https://launchpad.net/bugs/2163314>`__ diff --git a/releasenotes/notes/bug-2165921-a70d2839ab32f423.yaml b/releasenotes/notes/bug-2165921-a70d2839ab32f423.yaml new file mode 100644 index 000000000..91f900ec4 --- /dev/null +++ b/releasenotes/notes/bug-2165921-a70d2839ab32f423.yaml @@ -0,0 +1,7 @@ +--- +fixes: + - | + Install the Prometheus datasource plugin for Grafana 13.2+. + Grafana 13.2 moved Prometheus to a standalone plugin, which + is not automatically installed in offline environments. + `LP#2165921 <https://launchpad.net/bugs/2165921>`__ diff --git a/releasenotes/notes/bug-2166075-ea3b86dd40834b08.yaml b/releasenotes/notes/bug-2166075-ea3b86dd40834b08.yaml new file mode 100644 index 000000000..62dda913c --- /dev/null +++ b/releasenotes/notes/bug-2166075-ea3b86dd40834b08.yaml @@ -0,0 +1,5 @@ +--- +fixes: + - | + Replace deprecated ``grafana-cli`` with ``grafana cli``. + `LP#2166075 <https://launchpad.net/bugs/2166075>`__ diff --git a/releasenotes/notes/cleanup-repos-85faf33025fd7e5c.yaml b/releasenotes/notes/cleanup-repos-85faf33025fd7e5c.yaml new file mode 100644 index 000000000..627cce663 --- /dev/null +++ b/releasenotes/notes/cleanup-repos-85faf33025fd7e5c.yaml @@ -0,0 +1,16 @@ +--- +features: + - | + Repository cleanup is now performed via Jinja2 macros + (``kolla_cleanup_repos()``) injected directly into each Dockerfile + template, replacing the previous Python-level injection in + ``kolla_worker.py``. This ensures cleanup always runs before any + ``USER`` directive, avoiding permission errors when non-root service + users attempted to modify root-owned repository files. + - | + Base OS repositories (``baseos``, ``appstream`` on RPM distros; + ``debian``/``debian-security`` or ``ubuntu``/``ubuntu-security``/ + ``ubuntu-cloud-archive`` on Debian-based distros) are now explicitly + re-enabled in every service image via the new ``kolla_enable_base_repos()`` + macro. This guarantees that packages from base OS repositories remain + available in child images after parent image cleanup. diff --git a/releasenotes/notes/drop-centos10-c53c4eee6f8253d0.yaml b/releasenotes/notes/drop-centos10-c53c4eee6f8253d0.yaml new file mode 100644 index 000000000..69cd340e1 --- /dev/null +++ b/releasenotes/notes/drop-centos10-c53c4eee6f8253d0.yaml @@ -0,0 +1,6 @@ +--- +upgrade: + - | + Support for ``CentOS Stream 10`` has been dropped. Users using + ``base_centos_*`` override blocks should adapt them to + ``base_rocky_*``. diff --git a/releasenotes/notes/ironic-pxe-symlinks-18d09b19373f16c4.yaml b/releasenotes/notes/ironic-pxe-symlinks-18d09b19373f16c4.yaml new file mode 100644 index 000000000..1f4d4db37 --- /dev/null +++ b/releasenotes/notes/ironic-pxe-symlinks-18d09b19373f16c4.yaml @@ -0,0 +1,6 @@ +--- +fixes: + - | + Fixes ``ironic_pxe`` launch failures when symlinks to EFI files have + previously been created. + `LP#2160317 <https://launchpad.net/bugs/2160317>`__ diff --git a/releasenotes/notes/letsencrypt-dns01-challenge-432b5122dc832f4a.yaml b/releasenotes/notes/letsencrypt-dns01-challenge-432b5122dc832f4a.yaml new file mode 100644 index 000000000..0b35c17ca --- /dev/null +++ b/releasenotes/notes/letsencrypt-dns01-challenge-432b5122dc832f4a.yaml @@ -0,0 +1,14 @@ +--- +features: + - | + The ``letsencrypt-lego`` image now supports the ACME ``dns-01`` challenge + in addition to the existing ``http-01`` challenge. The + ``letsencrypt-certificates`` script accepts a new ``--challenge`` option + (defaulting to ``http-01``) and, when ``dns-01`` is selected, a + ``--dns-provider`` option naming any DNS provider supported by the bundled + ``lego`` binary. An optional ``--dns-additional-opts`` passthrough allows + forwarding extra ``lego`` DNS flags (such as ``--dns.resolvers`` or + ``--dns.propagation-disable-ans``) for environments where the default + authoritative name server propagation check cannot be performed. The + default behaviour is unchanged: without ``--challenge`` the ``http-01`` + challenge is used exactly as before. diff --git a/releasenotes/notes/prometheus-server-3.13LTS-b33f7bc56ed9cd44.yaml b/releasenotes/notes/prometheus-server-3.13LTS-b33f7bc56ed9cd44.yaml new file mode 100644 index 000000000..5dcd22e29 --- /dev/null +++ b/releasenotes/notes/prometheus-server-3.13LTS-b33f7bc56ed9cd44.yaml @@ -0,0 +1,4 @@ +--- +upgrade: + - | + Update Prometheus server to latest LTS release (3.5.x -> 3.13.x). diff --git a/releasenotes/notes/update-external-components-625902310f4f5536.yaml b/releasenotes/notes/update-external-components-625902310f4f5536.yaml new file mode 100644 index 000000000..bf69959f5 --- /dev/null +++ b/releasenotes/notes/update-external-components-625902310f4f5536.yaml @@ -0,0 +1,9 @@ +--- +upgrade: + - | + ``letsencrypt-lego`` has been updated from ``5.4.0`` to ``5.4.1``. + - | + ``prometheus-mtail`` has been updated from ``3.4.9`` to ``3.4.10``. + - | + ``prometheus-valkey-exporter`` has been updated from ``1.89.0`` to + ``1.90.0``. diff --git a/releasenotes/notes/update-external-components-7ad94672447b4901.yaml b/releasenotes/notes/update-external-components-7ad94672447b4901.yaml new file mode 100644 index 000000000..c6bace2f9 --- /dev/null +++ b/releasenotes/notes/update-external-components-7ad94672447b4901.yaml @@ -0,0 +1,35 @@ +--- +upgrade: + - | + ``etcd`` has been updated from ``3.6.5`` to ``3.6.14``. + - | + ``letsencrypt-lego`` has been updated from ``4.27.0`` to ``5.4.0``. + - | + ``magnum-conductor-plugin-helm`` has been updated from ``3.19.0`` to + ``4.2.4``. + - | + ``prometheus-alertmanager`` has been updated from ``0.31.1`` to ``0.34.0``. + - | + ``prometheus-cadvisor`` has been updated from ``0.56.2`` to ``0.60.5``. + - | + ``prometheus-elasticsearch-exporter`` has been updated from ``1.10.0`` to + ``1.11.0``. + - | + ``prometheus-libvirt-exporter`` has been updated from ``2.3.1`` to + ``2.5.0``. + - | + ``prometheus-memcached-exporter`` has been updated from ``0.15.5`` to + ``0.17.0``. + - | + ``prometheus-mtail`` has been updated from ``3.4.0`` to ``3.4.9``. + - | + ``prometheus-mysqld-exporter`` has been updated from ``0.18.0`` to + ``0.20.0``. + - | + ``prometheus-node-exporter`` has been updated from ``1.10.2`` to + ``1.12.1``. + - | + ``prometheus-server`` has been updated from ``3.5.4`` to ``3.5.5``. + - | + ``prometheus-valkey-exporter`` has been updated from ``1.82.0`` to + ``1.89.0``. diff --git a/releasenotes/notes/update-external-components-94f3c485be382c8e.yaml b/releasenotes/notes/update-external-components-94f3c485be382c8e.yaml new file mode 100644 index 000000000..e72f54e25 --- /dev/null +++ b/releasenotes/notes/update-external-components-94f3c485be382c8e.yaml @@ -0,0 +1,7 @@ +--- +upgrade: + - | + ``prometheus-mtail`` has been updated from ``3.4.10`` to ``3.4.11``. + - | + ``prometheus-openstack-network-exporter`` has been updated from ``0.2.0`` + to ``0.3.0``. diff --git a/releasenotes/notes/update-external-components-97b947c8fdb88ac9.yaml b/releasenotes/notes/update-external-components-97b947c8fdb88ac9.yaml new file mode 100644 index 000000000..d758f9412 --- /dev/null +++ b/releasenotes/notes/update-external-components-97b947c8fdb88ac9.yaml @@ -0,0 +1,7 @@ +--- +upgrade: + - | + ``prometheus-server`` has been updated from ``3.13.2`` to ``3.13.3``. + - | + ``prometheus-valkey-exporter`` has been updated from ``1.90.0`` to + ``1.91.1``. diff --git a/releasenotes/notes/update-magnum-conductor-plugin-helm-6afa6c8dc5a98d5a.yaml b/releasenotes/notes/update-magnum-conductor-plugin-helm-6afa6c8dc5a98d5a.yaml new file mode 100644 index 000000000..122ee7c67 --- /dev/null +++ b/releasenotes/notes/update-magnum-conductor-plugin-helm-6afa6c8dc5a98d5a.yaml @@ -0,0 +1,5 @@ +--- +upgrade: + - | + ``magnum-conductor-plugin-helm`` has been updated from ``4.2.4`` to + ``4.3.0``. diff --git a/releasenotes/notes/update-prometheus-mtail-4d73dd09f088a408.yaml b/releasenotes/notes/update-prometheus-mtail-4d73dd09f088a408.yaml new file mode 100644 index 000000000..a9c6e7ed8 --- /dev/null +++ b/releasenotes/notes/update-prometheus-mtail-4d73dd09f088a408.yaml @@ -0,0 +1,4 @@ +--- +upgrade: + - | + ``prometheus-mtail`` has been updated from ``3.4.11`` to ``3.4.12``. diff --git a/roles/kolla-build-config/defaults/main.yml b/roles/kolla-build-config/defaults/main.yml index f00b61190..ced1ac659 100644 --- a/roles/kolla-build-config/defaults/main.yml +++ b/roles/kolla-build-config/defaults/main.yml @@ -11,65 +11,70 @@ kolla_build_config: {} # Dict mapping project names to sections in kolla-build.conf. kolla_build_sources: - openstack/requirements: openstack-base - + gnocchixyz/gnocchi: gnocchi-base openstack/aodh: aodh-base openstack/barbican: barbican-base openstack/bifrost: bifrost-base openstack/blazar: blazar-base + openstack/blazar-dashboard: horizon-plugin-blazar-dashboard openstack/blazar-nova: nova-base-plugin-blazar openstack/ceilometer: ceilometer-base openstack/cinder: cinder-base openstack/cloudkitty: cloudkitty-base + openstack/cloudkitty-dashboard: horizon-plugin-cloudkitty-dashboard + openstack/cyborg: cyborg-base openstack/designate: designate-base + openstack/designate-dashboard: horizon-plugin-designate-dashboard openstack/glance: glance-base - openstack/gnocchi: gnocchi-base openstack/heat: heat-base + openstack/heat-dashboard: horizon-plugin-heat-dashboard + openstack/horizon: horizon openstack/ironic: ironic-base + openstack/ironic-prometheus-exporter: ironic-conductor-plugin-prometheus-exporter + openstack/ironic-ui: horizon-plugin-ironic-ui openstack/keystone: keystone-base openstack/magnum: magnum-base + openstack/magnum-ui: horizon-plugin-magnum-ui openstack/manila: manila-base + openstack/manila-ui: horizon-plugin-manila-ui + openstack/masakari: masakari-base + openstack/masakari-dashboard: horizon-plugin-masakari-dashboard + openstack/masakari-monitors: masakari-monitors openstack/mistral: mistral-base - openstack/monasca-agent: monasca-agent - openstack/monasca-api: monasca-api - openstack/monasca-notification: monasca-notification - openstack/monasca-persister: monasca-persister - openstack/monasca-statsd: monasca-statsd - openstack/monasca-common: monasca-thresh-additions-monasca-common - openstack/neutron: neutron-base + openstack/mistral-dashboard: horizon-plugin-mistral-dashboard openstack/networking-baremetal: neutron-base-plugin-networking-baremetal - openstack/neutron-dynamic-routing: - - neutron-bgp-dragent - - neutron-server-plugin-neutron-dynamic-routing openstack/networking-generic-switch: neutron-base-plugin-networking-generic-switch openstack/networking-sfc: - neutron-base-plugin-networking-sfc - tacker-base-plugin-networking-sfc + openstack/neutron: neutron-base + openstack/neutron-dynamic-routing: + - neutron-bgp-dragent + - neutron-server-plugin-neutron-dynamic-routing + openstack/neutron-fwaas: neutron-base-plugin-neutron-fwaas + openstack/neutron-fwaas-dashboard: horizon-plugin-fwaas-dashboard openstack/neutron-vpnaas: - neutron-vpnaas-agent - neutron-base-plugin-vpnaas-agent + openstack/neutron-vpnaas-dashboard: horizon-plugin-neutron-vpnaas-dashboard openstack/nova: nova-base openstack/octavia: octavia-base - openstack/skyline: skyline-base + openstack/octavia-dashboard: horizon-plugin-octavia-dashboard + openstack/ovn-octavia-provider: + - octavia-api-plugin-ovn-octavia-provider + - octavia-driver-agent-plugin-ovn-octavia-provider + openstack/placement: placement-base + openstack/requirements: + - openstack-base + - kolla-toolbox + openstack/skyline-apiserver: skyline-apiserver + openstack/skyline-console: skyline-console openstack/tacker: - tacker-base - mistral-base-plugin-tacker - openstack/trove: trove-base - openstack/watcher: watcher-base - - # Horizon and its plugins - openstack/horizon: horizon - openstack/blazar-dashboard: horizon-plugin-blazar-dashboard - openstack/cloudkitty-dashboard: horizon-plugin-cloudkitty-dashboard - openstack/designate-dashboard: horizon-plugin-designate-dashboard - openstack/heat-dashboard: horizon-plugin-heat-dashboard - openstack/ironic-ui: horizon-plugin-ironic-ui - openstack/magnum-ui: horizon-plugin-magnum-ui - openstack/manila-ui: horizon-plugin-manila-ui - openstack/mistral-dashboard: horizon-plugin-mistral-dashboard - openstack/monasca-ui: horizon-plugin-monasca-ui - openstack/neutron-vpnaas-dashboard: horizon-plugin-neutron-vpnaas-dashboard - openstack/octavia-dashboard: horizon-plugin-octavia-dashboard openstack/tacker-horizon: horizon-plugin-tacker-dashboard + openstack/tap-as-a-service: neutron-base-plugin-tap-as-a-service + openstack/trove: trove-base openstack/trove-dashboard: horizon-plugin-trove-dashboard + openstack/watcher: watcher-base openstack/watcher-dashboard: horizon-plugin-watcher-dashboard diff --git a/roles/kolla-build-config/templates/kolla-build.conf.j2 b/roles/kolla-build-config/templates/kolla-build.conf.j2 index bfc5dd426..eddbdab14 100644 --- a/roles/kolla-build-config/templates/kolla-build.conf.j2 +++ b/roles/kolla-build-config/templates/kolla-build.conf.j2 @@ -7,21 +7,24 @@ {% endfor %} {% endfor %} -{# Generate a section for every build source for which there is a Zuul change. #} +{# Generate a section for every build source Zuul has a checkout for, + whether it's a required-project or a Depends-On. #} -{% for zuul_item in zuul['items'] %} -{% if zuul_item.project.name in kolla_build_sources %} -{% set source_names = kolla_build_sources[zuul_item.project.name] %} +{% if not (is_upgrade | default(False) | bool) %} +{% for canonical_name, project in zuul['projects'].items() %} +{% if project.name in kolla_build_sources %} +{% set source_names = kolla_build_sources[project.name] %} {% if source_names is string %} [{{ source_names }}] type = local -location = {{ ansible_env.PWD ~ '/' ~ zuul_item.project.src_dir }} +location = {{ ansible_env.PWD ~ '/' ~ project.src_dir }} {% else %} {% for source_name in source_names %} [{{ source_name }}] type = local -location = {{ ansible_env.PWD ~ '/' ~ zuul_item.project.src_dir }} +location = {{ ansible_env.PWD ~ '/' ~ project.src_dir }} {% endfor %} {% endif %} {% endif %} {% endfor %} +{% endif %} diff --git a/tests/files/process_build_logs.py b/tests/files/process_build_logs.py index 0db5228d9..248acf954 100755 --- a/tests/files/process_build_logs.py +++ b/tests/files/process_build_logs.py @@ -30,7 +30,7 @@ parser.add_argument('-b', '--base', required=True) args = vars(parser.parse_args()) -if args['base'] not in ['rocky', 'centos']: +if args['base'] != 'rocky': print("Non rpm-based distros are not yet supported.") sys.exit() diff --git a/tests/templates/repos.yaml.j2 b/tests/templates/repos.yaml.j2 index 42c6724f9..573173d95 100644 --- a/tests/templates/repos.yaml.j2 +++ b/tests/templates/repos.yaml.j2 @@ -2,12 +2,14 @@ {% if base_distro == 'debian' %} debian: debian: + build_only: true component: "main" gpg_key: "/usr/share/keyrings/debian-archive-keyring.gpg" suite: "trixie trixie-updates trixie-backports" trusted: True url: "http://{{ nodepool_mirror_host }}/debian" debian-security: + build_only: true component: "main" gpg_key: "/usr/share/keyrings/debian-archive-keyring.gpg" suite: "trixie-security" @@ -17,31 +19,42 @@ debian: rocky: appstream: baseurl: "http://dl.rockylinux.org/$contentdir/$releasever/AppStream/$basearch/os/" + build_only: true gpgkey: "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10" name: "appstream" baseos: baseurl: "http://dl.rockylinux.org/$contentdir/$releasever/BaseOS/$basearch/os/" + build_only: true gpgkey: "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10" name: "baseos" crb: baseurl: "http://dl.rockylinux.org/$contentdir/$releasever/CRB/$basearch/os/" + build_only: true gpgkey: "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-10" name: "crb" + openvswitch: + baseurl: "http://centos-stream.mirrorservice.org/SIGs/$releasever-stream/nfv/$basearch/openvswitch-2/" + build_only: true + gpgkey: "https://www.centos.org/keys/RPM-GPG-KEY-CentOS-SIG-NFV" + name: "centos-nfv-openvswitch" {% elif base_distro == 'ubuntu' %} ubuntu: ubuntu: + build_only: true component: "main universe" gpg_key: "/usr/share/keyrings/ubuntu-archive-keyring.gpg" suite: "noble noble-updates noble-backports" trusted: True url: "http://{{ nodepool_mirror_host }}/ubuntu/" ubuntu-security: + build_only: true component: "main universe" gpg_key: "/usr/share/keyrings/ubuntu-archive-keyring.gpg" suite: "noble-security" trusted: True url: "http://{{ nodepool_mirror_host }}/ubuntu/" ubuntu-cloud-archive: + build_only: true component: "main" gpg_key: "/usr/share/keyrings/ubuntu-cloud-keyring.gpg" suite: "noble-updates/gazpacho" @@ -49,12 +62,14 @@ ubuntu: url: "http://{{ nodepool_mirror_host }}/ubuntu-cloud-archive" ubuntu-aarch64: ubuntu: + build_only: true component: "main universe" gpg_key: "/usr/share/keyrings/ubuntu-archive-keyring.gpg" suite: "noble noble-updates noble-backports" trusted: True url: "http://{{ nodepool_mirror_host }}/ubuntu-ports/" ubuntu-security: + build_only: true component: "main universe" gpg_key: "/usr/share/keyrings/ubuntu-archive-keyring.gpg" suite: "noble-security" diff --git a/tests/vars/zuul.yml b/tests/vars/zuul.yml index dbcc01d19..b2962b85b 100644 --- a/tests/vars/zuul.yml +++ b/tests/vars/zuul.yml @@ -3,7 +3,6 @@ nodepool_rdo_proxy: "http://{{ zuul_site_mirror_fqdn }}:8080/rdo" nodepool_pypi_mirror: "http://{{ zuul_site_mirror_fqdn }}:8080/pypi/simple" nodepool_mirror_host: "{{ zuul_site_mirror_fqdn }}" nodepool_npmjs_proxy: "http://{{ zuul_site_mirror_fqdn }}:8080/registry.npmjs/" -nodepool_cbs_centos_proxy: "http://{{ zuul_site_mirror_fqdn }}:8080/cbs.centos" nodepool_docker_proxy: "http://{{ zuul_site_mirror_fqdn }}:8080/docker" # NOTE(hrw): wheel cache goes over 80/443 not on 8080 diff --git a/tools/check_sources.py b/tools/check_sources.py index 189ddb170..38fc03acc 100644 --- a/tools/check_sources.py +++ b/tools/check_sources.py @@ -19,10 +19,7 @@ import sys import textwrap import requests - -sys.path.append(os.getcwd()) - -from kolla.common.sources import SOURCES # noqa: E402 +import yaml def github_headers(): @@ -202,14 +199,9 @@ def get_helm_sh_hashes(version, arch_map): return hashes -SOURCES_PATH = os.path.normpath( - os.path.join(os.path.dirname(os.path.abspath(__file__)), - '..', 'kolla', 'common', 'sources.py')) - - -def apply_update(name, info, new_v, new_hashes): - """Update version and sha256 entries for one source in sources.py.""" - with open(SOURCES_PATH) as f: +def apply_update(name, info, new_v, new_hashes, sources_path): + """Update version and sha256 entries for one source in sources.yaml.""" + with open(sources_path) as f: content = f.read() version_key = 'reference' if info.get('type') == 'git' else 'version' @@ -217,26 +209,23 @@ def apply_update(name, info, new_v, new_hashes): if old_v: new_v_str = ('v' + new_v) if old_v.startswith('v') else new_v content = content.replace( - f"'{version_key}': '{old_v}'", - f"'{version_key}': '{new_v_str}'", 1) + f'{version_key}: "{old_v}"', + f'{version_key}: "{new_v_str}"', 1) for arch, new_sha in new_hashes.items(): old_sha = info.get('sha256', {}).get(arch, '') if old_sha and old_sha != new_sha: content = content.replace(old_sha, new_sha) - with open(SOURCES_PATH, 'w') as f: + with open(sources_path, 'w') as f: f.write(content) -REPO_ROOT = os.path.normpath( - os.path.join(os.path.dirname(os.path.abspath(__file__)), '..')) - - -def create_releasenote(changes): +def create_releasenote(changes, repo_root): """Create a reno release note listing all updated components. :param changes: list of (name, old_version, new_version) tuples + :param repo_root: root of the kolla repository checkout """ if len(changes) == 1: slug = f"update-{changes[0][0]}" @@ -244,7 +233,7 @@ def create_releasenote(changes): slug = "update-external-components" result = subprocess.run( # nosec - ['reno', 'new', slug, REPO_ROOT], + ['reno', 'new', slug, repo_root], capture_output=True, text=True) if result.returncode != 0: @@ -269,8 +258,13 @@ def create_releasenote(changes): print(f"Created release note: {note_path}") -def check_versions(update=False): - """Monitor Kolla architectures and compare with latest releases.""" +def check_versions(sources, sources_path, repo_root, update=False): + """Monitor Kolla architectures and compare with latest releases. + + :param sources: the kolla.common.sources.SOURCES dict + :param sources_path: path to the sources.yaml file to update + :param repo_root: root of the kolla repository checkout + """ check_github_rate_limit() arch_map = {'amd64': 'linux-amd64', 'arm64': 'linux-arm64'} version_changes = [] @@ -280,7 +274,7 @@ def check_versions(update=False): print(header) print("-" * 110) - for name, info in SOURCES.items(): + for name, info in sources.items(): current_v = info.get('version') or info.get('reference', '') if not re.match(r'^v?\d+[\.\d]+', current_v or ''): current_v = None @@ -324,23 +318,54 @@ def check_versions(update=False): print(line) if update and lines_to_print: - apply_update(name, info, latest_v, new_hashes) + apply_update(name, info, latest_v, new_hashes, + sources_path) if status_v == "UPDATE": version_changes.append( (name, current_v, latest_v)) if update and version_changes: - create_releasenote(version_changes) + create_releasenote(version_changes, repo_root) -if __name__ == "__main__": +def parse_args(): parser = argparse.ArgumentParser( description='Check and optionally update Kolla source versions.') + parser.add_argument('--repo', default=None, + help='Path to the kolla repository checkout ' + '(default: the checkout this script lives in, ' + 'if any, otherwise the current directory)') parser.add_argument('--update', action='store_true', - help='Update kolla/common/sources.py in place') - args = parser.parse_args() + help='Update kolla/common/sources.yaml in place') + return parser.parse_args() + + +def default_repo_root(): + """Guess the repo root when run in-tree from <repo>/tools/.""" + in_tree = os.path.normpath( + os.path.join(os.path.dirname(os.path.abspath(__file__)), '..')) + if os.path.isfile( + os.path.join(in_tree, 'kolla', 'common', 'sources.yaml')): + return in_tree + return os.getcwd() + + +if __name__ == "__main__": + args = parse_args() if args.update and not shutil.which('reno'): print("ERROR: 'reno' is not installed or not in PATH. " "Install it with: pip install reno") sys.exit(1) - check_versions(update=args.update) + + repo_root = os.path.abspath(args.repo or default_repo_root()) + sources_path = os.path.join( + repo_root, 'kolla', 'common', 'sources.yaml') + if not os.path.isfile(sources_path): + print(f"ERROR: {sources_path} not found. Pass --repo pointing at " + "a kolla repository checkout.") + sys.exit(1) + + with open(sources_path) as f: + SOURCES = yaml.safe_load(f) + + check_versions(SOURCES, sources_path, repo_root, update=args.update) diff --git a/tools/validate-build-sources-overrides.py b/tools/validate-build-sources-overrides.py new file mode 100644 index 000000000..5d9db62f5 --- /dev/null +++ b/tools/validate-build-sources-overrides.py @@ -0,0 +1,175 @@ +#!/usr/bin/env python3 + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Check that every buildable-from-source entry in kolla/common/sources.yaml +has a matching CI override in +roles/kolla-build-config/defaults/main.yml (kolla_build_sources), that +each such project is declared in the kolla-base job's required-projects +in zuul.d/base.yaml, and that any pinned git reference in sources.yaml +matches the override-checkout used to fetch it in CI. This ensures CI +builds every source from a Zuul-cached git checkout instead of +downloading a tarball from tarballs.opendev.org. +""" + +import os +import sys + +import yaml + +PROJECT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), '..')) +sys.path.append(PROJECT_ROOT) + +from kolla.common.sources import SOURCES # noqa: E402 + +BUILD_SOURCES_PATH = os.path.join( + PROJECT_ROOT, 'roles', 'kolla-build-config', 'defaults', 'main.yml') +ZUUL_BASE_PATH = os.path.join(PROJECT_ROOT, 'zuul.d', 'base.yaml') + +# Sources deliberately not covered by kolla_build_sources, with the +# reason a git-checkout override doesn't apply to them. +EXCLUDED_SOURCES = { + # Binary releases downloaded from GitHub, not built from a source + # checkout. + 'etcd', + 'letsencrypt-lego', + 'magnum-conductor-plugin-helm', + # GitHub projects that are not registered with the OpenStack Zuul + # tenant (unlike gnocchixyz/gnocchi), so they can't be required-projects. + 'kerbside-base', + 'mariadb-server-plugin-mariadb-docker', +} + + +def is_excluded(name): + return name in EXCLUDED_SOURCES or name.startswith('prometheus-') + + +def as_list(section_names): + if isinstance(section_names, str): + return [section_names] + return list(section_names) + + +def load_kolla_build_sources(): + with open(BUILD_SOURCES_PATH) as f: + return yaml.safe_load(f)['kolla_build_sources'] + + +def load_required_projects(): + """Parse the kolla-base job's required-projects in zuul.d/base.yaml. + + Returns (all project names, {project: override-checkout}). + """ + with open(ZUUL_BASE_PATH) as f: + docs = yaml.safe_load(f) + + job = next(doc['job'] for doc in docs + if 'job' in doc and doc['job']['name'] == 'kolla-base') + + all_projects = set() + checkouts = {} + for entry in job.get('required-projects', []): + if isinstance(entry, str): + all_projects.add(entry) + else: + all_projects.add(entry['name']) + if 'override-checkout' in entry: + checkouts[entry['name']] = entry['override-checkout'] + return all_projects, checkouts + + +def main(): + kolla_build_sources = load_kolla_build_sources() + required_projects, override_checkouts = load_required_projects() + + covered = set() + for section_names in kolla_build_sources.values(): + covered.update(as_list(section_names)) + + buildable = {name for name in SOURCES if not is_excluded(name)} + + missing = sorted(buildable - covered) + stale = sorted(covered - set(SOURCES)) + redundant_exclusions = sorted( + name for name in EXCLUDED_SOURCES if name not in SOURCES) + not_required = sorted(set(kolla_build_sources) - required_projects) + + ref_errors = [] + for project, section_names in sorted(kolla_build_sources.items()): + for section in as_list(section_names): + info = SOURCES.get(section) + if not info or info.get('type') != 'git': + continue + expected_ref = info.get('reference') + actual = override_checkouts.get(project) + if actual is None: + ref_errors.append( + f"{project} ({section}) is pinned to reference " + f"'{expected_ref}' in sources.yaml but has no " + "override-checkout in zuul.d/base.yaml " + "required-projects") + elif actual != expected_ref: + ref_errors.append( + f"{project} ({section}): sources.yaml reference " + f"'{expected_ref}' != required-projects " + f"override-checkout '{actual}'") + + for project, checkout in sorted(override_checkouts.items()): + section_names = kolla_build_sources.get(project) + if section_names is None: + continue + git_sections = [s for s in as_list(section_names) + if SOURCES.get(s, {}).get('type') == 'git'] + if not git_sections: + ref_errors.append( + f"{project} has override-checkout '{checkout}' in " + "required-projects but none of its kolla_build_sources " + "sections are type 'git' in sources.yaml") + + if missing: + print("ERROR: sources.yaml entries missing a kolla_build_sources " + "override in roles/kolla-build-config/defaults/main.yml:") + for name in missing: + print(f" {name}") + + if stale: + print("ERROR: kolla_build_sources overrides referencing sections " + "that no longer exist in kolla/common/sources.yaml:") + for name in stale: + print(f" {name}") + + if redundant_exclusions: + print("ERROR: EXCLUDED_SOURCES entries referencing sections that " + "no longer exist in kolla/common/sources.yaml, remove them:") + for name in redundant_exclusions: + print(f" {name}") + + if not_required: + print("ERROR: kolla_build_sources projects missing from the " + "kolla-base job's required-projects in zuul.d/base.yaml:") + for name in not_required: + print(f" {name}") + + if ref_errors: + print("ERROR: git reference mismatches between sources.yaml and " + "zuul.d/base.yaml required-projects:") + for msg in ref_errors: + print(f" {msg}") + + return 1 if (missing or stale or redundant_exclusions or not_required + or ref_errors) else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/version-check.py b/tools/version-check.py index 2220b4a52..d2b04daea 100755 --- a/tools/version-check.py +++ b/tools/version-check.py @@ -41,7 +41,6 @@ TARGET = '.releases' SKIP_PROJECTS = { 'gnocchi-base': 'Gnocchi is not managed by openstack/releases project', - 'monasca-thresh': 'Package not published in tarballs.openstack.org', } # NOTE(hrw): those projects we take as they are they may have just one old @@ -52,6 +52,7 @@ commands = flake8 {posargs} bash {toxinidir}/tools/validate-all-dockerfiles.sh python {toxinidir}/tools/validate-all-file.py + python {toxinidir}/tools/validate-build-sources-overrides.py bandit -r docker kolla tests tools codespell -I {toxinidir}/.codespell-ignore -x zuul.d/secrets.yaml ansible-galaxy collection install -r test-requirements.yml diff --git a/zuul.d/base.yaml b/zuul.d/base.yaml index 8d58c81f0..21cc98371 100644 --- a/zuul.d/base.yaml +++ b/zuul.d/base.yaml @@ -10,6 +10,63 @@ attempts: 5 roles: - zuul: openstack/ansible-collection-kolla + required-projects: + - name: gnocchixyz/gnocchi + override-checkout: 4.7.0 + - openstack/aodh + - openstack/barbican + - openstack/bifrost + - openstack/blazar + - openstack/blazar-dashboard + - openstack/blazar-nova + - openstack/ceilometer + - openstack/cinder + - openstack/cloudkitty + - openstack/cloudkitty-dashboard + - openstack/cyborg + - openstack/designate + - openstack/designate-dashboard + - openstack/glance + - openstack/heat + - openstack/heat-dashboard + - openstack/horizon + - openstack/ironic + - openstack/ironic-prometheus-exporter + - openstack/ironic-ui + - openstack/keystone + - openstack/magnum + - openstack/magnum-ui + - openstack/manila + - openstack/manila-ui + - openstack/masakari + - openstack/masakari-dashboard + - openstack/masakari-monitors + - openstack/mistral + - openstack/mistral-dashboard + - openstack/networking-baremetal + - openstack/networking-generic-switch + - openstack/networking-sfc + - openstack/neutron + - openstack/neutron-dynamic-routing + - openstack/neutron-fwaas + - openstack/neutron-fwaas-dashboard + - openstack/neutron-vpnaas + - openstack/neutron-vpnaas-dashboard + - openstack/nova + - openstack/octavia + - openstack/octavia-dashboard + - openstack/ovn-octavia-provider + - openstack/placement + - openstack/requirements + - openstack/skyline-apiserver + - openstack/skyline-console + - openstack/tacker + - openstack/tacker-horizon + - openstack/tap-as-a-service + - openstack/trove + - openstack/trove-dashboard + - openstack/watcher + - openstack/watcher-dashboard irrelevant-files: - ^.*\.md$ - ^.*\.rst$ diff --git a/zuul.d/centos.yaml b/zuul.d/centos.yaml deleted file mode 100644 index 3c48f9e0c..000000000 --- a/zuul.d/centos.yaml +++ /dev/null @@ -1,31 +0,0 @@ ---- -- job: - name: kolla-build-centos-10s - parent: kolla-base - nodeset: kolla-centos-10s-8GB - voting: false - -- job: - name: kolla-build-centos-10s-arm64 - parent: kolla-build-centos-10s - nodeset: kolla-centos-10s-arm64-8GB - voting: false - -- job: - name: kolla-build-centos-10s-no-infra-wheels - parent: kolla-build-no-infra-wheels-base - nodeset: kolla-centos-10s-8GB - -- project-template: - name: kolla-build-centos - description: | - Runs Kolla CentOS build jobs. - check: - jobs: - - kolla-build-centos-10s - check-arm64: - jobs: - - kolla-build-centos-10s-arm64 - experimental: - jobs: - - kolla-build-centos-10s-no-infra-wheels diff --git a/zuul.d/nodesets.yaml b/zuul.d/nodesets.yaml index 1c85b45fb..ccb3513f0 100644 --- a/zuul.d/nodesets.yaml +++ b/zuul.d/nodesets.yaml @@ -1,17 +1,5 @@ --- - nodeset: - name: kolla-centos-10s-8GB - nodes: - - name: primary - label: centos-10-stream-8GB - -- nodeset: - name: kolla-centos-10s-arm64-8GB - nodes: - - name: primary - label: centos-10-stream-arm64-8GB - -- nodeset: name: kolla-debian-bookworm-8GB nodes: - name: primary diff --git a/zuul.d/project.yaml b/zuul.d/project.yaml index bc7d58876..782a5c640 100644 --- a/zuul.d/project.yaml +++ b/zuul.d/project.yaml @@ -5,7 +5,6 @@ - project: templates: - check-requirements - - kolla-build-centos - kolla-build-debian - kolla-build-rocky - kolla-build-ubuntu @@ -15,10 +14,12 @@ - kolla-scenario-cells - kolla-scenario-cephadm - kolla-scenario-ironic + - kolla-scenario-lets-encrypt - kolla-scenario-mariadb - kolla-scenario-masakari - kolla-scenario-octavia - kolla-scenario-ovn + - kolla-scenario-prometheus-opensearch - kolla-scenario-telemetry - kolla-tox-genconfig - openstack-python3-jobs diff --git a/zuul.d/scenarios/aio.yaml b/zuul.d/scenarios/aio.yaml index 4a8a324ea..0d2258848 100644 --- a/zuul.d/scenarios/aio.yaml +++ b/zuul.d/scenarios/aio.yaml @@ -5,9 +5,10 @@ Runs Kolla-Ansible AIO jobs on all Kolla changes. check: jobs: - - kolla-ansible-centos-10s: &files - files: ^kolla/docker/ - - kolla-ansible-debian-trixie: *files + - kolla-ansible-debian-trixie: &files + files: + - ^kolla/common/sources.yaml + - ^kolla/docker/ - kolla-ansible-debian-trixie-upgrade: *files - kolla-ansible-rocky-10: *files - kolla-ansible-rocky-10-upgrade: *files @@ -15,7 +16,6 @@ - kolla-ansible-ubuntu-noble-upgrade: *files check-arm64: jobs: - - kolla-ansible-centos-10s-aarch64: *files - kolla-ansible-debian-trixie-aarch64: *files gate: jobs: diff --git a/zuul.d/scenarios/lets-encrypt.yaml b/zuul.d/scenarios/lets-encrypt.yaml new file mode 100644 index 000000000..9649d4ea6 --- /dev/null +++ b/zuul.d/scenarios/lets-encrypt.yaml @@ -0,0 +1,12 @@ +--- +- project-template: + name: kolla-scenario-lets-encrypt + description: | + Runs Kolla-Ansible Let's Encrypt scenario jobs. + check: + jobs: + - kolla-ansible-debian-trixie-lets-encrypt: &files + files: + - ^kolla/common/sources.yaml + - ^kolla/docker/(haproxy|httpd|letsencrypt)/ + - kolla-ansible-ubuntu-noble-lets-encrypt: *files diff --git a/zuul.d/scenarios/mariadb.yaml b/zuul.d/scenarios/mariadb.yaml index 882e1338e..c53a27fca 100644 --- a/zuul.d/scenarios/mariadb.yaml +++ b/zuul.d/scenarios/mariadb.yaml @@ -6,6 +6,8 @@ check: jobs: - kolla-ansible-debian-trixie-mariadb: &files - files: ^kolla/docker/(base|mariadb-server)/ + files: + - ^kolla/common/sources.yaml + - ^kolla/docker/(base|mariadb-server)/ - kolla-ansible-rocky-10-mariadb: *files - kolla-ansible-ubuntu-noble-mariadb: *files diff --git a/zuul.d/scenarios/prometheus-opensearch.yaml b/zuul.d/scenarios/prometheus-opensearch.yaml new file mode 100644 index 000000000..c1a5778de --- /dev/null +++ b/zuul.d/scenarios/prometheus-opensearch.yaml @@ -0,0 +1,14 @@ +--- +- project-template: + name: kolla-scenario-prometheus-opensearch + check: + jobs: + - kolla-ansible-debian-trixie-prometheus-opensearch: &files + files: + - ^kolla/common/sources.yaml + - ^kolla/docker/(base|fluentd|grafana|opensearch|prometheus)/ + - kolla-ansible-debian-trixie-prometheus-opensearch-upgrade: *files + - kolla-ansible-rocky-10-prometheus-opensearch: *files + - kolla-ansible-rocky-10-prometheus-opensearch-upgrade: *files + - kolla-ansible-ubuntu-noble-prometheus-opensearch: *files + - kolla-ansible-ubuntu-noble-prometheus-opensearch-upgrade: *files |