summaryrefslogtreecommitdiff
path: root/kolla/tests
AgeCommit message (Collapse)AuthorFilesLines
2023-09-29rabbitmq: Use novemberain repos for Debian/UbuntuMichal Nasiadka1-1/+23
CloudSmith is over budget, see [1]. As mentionedin in [2] - we're using amd64 packages on aarch64, since the community mirror does not mirror aarch64 - and the packages should be really marked as noarch. [3] has also been incorporated in this backport [1]: https://github.com/rabbitmq/rabbitmq-server/discussions/9526 [2]: https://github.com/rabbitmq/rabbitmq-server/discussions/9553 [3]: https://review.opendev.org/c/openstack/kolla/+/896927 Closes-Bug: #2037469 Change-Id: I83179ce5d53fed6cfe7f5fcbf6fdecd7199baa36 (cherry picked from commit 47b593d1e5531c693dde3e111ed3be91c60c41be)
2023-03-31Fix test malicious tarball failMichal Nasiadka1-0/+72
Since I650fcbc8f773fad8116338f6fb0cf7b4f4f17b33 builds from git fails on plugins with an exception: 'tarfile.ReadError: not a gzip file' because the test checks only gzip compressed archives but plugins created as plain tar files. This change fixes the issue using transparent compression support and also adds some debug info. Closes-Bug: #1990432 Change-Id: If0f9b4dd058a257d0653332d1b663e150c717304 Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com> Co-Authored-by: Michal Nasiadka <mnasiadka@gmail.com> (cherry picked from commit 143765fb67221cc51f1dc56a41ac2b67dddc453f)
2023-03-20Test for unsafe files in tarfile.extractallMichal Nasiadka1-0/+42
In addition to that, mark bifrost unbuildable since it's failing on EPEL 9 enablement. Closes-Bug: #1990432 Change-Id: I650fcbc8f773fad8116338f6fb0cf7b4f4f17b33 (cherry picked from commit 3d008b7f5ec2a54d004e8e9370f303ef9dc7858b)
2022-12-06Drop elasticsearch, kibana and logstashMichal Nasiadka1-9/+9
Support has been dropped in kolla-ansible, the upgrade path is opensearch. Change-Id: I6f32aacb4e0b7dd1eb7206e02425b16f5beae4f0
2022-10-06Move KollaWorker stuff to separate fileMarcin Juszkiewicz1-7/+8
As part of kolla/image/ refactoring we move KollaWorker related classes into own file. Based on patch by Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Change-Id: I55c3f6b34631048b2463b02e6ea611d7e4c1a912
2022-10-06Move build related tasks to separate fileMarcin Juszkiewicz1-14/+15
As part of kolla/image/ refactoring we move build related task classes into own file. Based on patch by Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Change-Id: I093620679016b37e1664c9fe4cf7559433e744b7
2022-10-06Move status and log related things to separate fileMarcin Juszkiewicz1-37/+38
As part of kolla/image/ refactoring we move status enums/consts into own file. Based on patch by Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Change-Id: I940816c7e487f6d5c7abc4446226e2992cf9e63d
2022-09-27Move to CentOS Stream 9Marcin Juszkiewicz2-5/+4
OpenStack 'zed' requires Python 3.8+ so RHEL 8 family has to go. This changeset moves to CentOS Stream 9 while move to RockyLinux 9 is planned as final solution. CI moved to CentOS Stream 9 nodes. Depends-on: https://review.opendev.org/c/openstack/kolla-ansible/+/839715 Change-Id: I113b9984294cf8663d3fc0c8840320e1d40ea731
2022-08-22Bump minimum docker-py versionDr. Jens Harbott1-16/+0
With version 6.0.0 of the docker python module, their implementation of versioning has changed, making our check for old versions fail. Since version 3.0.0 is from 2018, we should be safe to assume that as a minimum version though, so we can just get rid of the special handling of older versions. Change-Id: I077b7b5acf2c1f9beb6da06d3555e2ebe30831d1
2022-07-27Move to Ubuntu Jammy (22.04)Michal Nasiadka1-1/+1
mariadb - downloads.mariadb.com doesn't have jammy version, should be there soon - using Ubuntu provided version for now telegraf/collectd - collectd packages missing in Jammy - marked as unbuildable monasca - requires new librdkafka, no wheel for Python 3.10 for now Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/846070 Change-Id: I11b4fbcaba165824252902d0270ce1dbbc3d6e9d
2022-04-14drop install_type almost everywhereMarcin Juszkiewicz1-11/+0
The only place where it stays is argument parsing where it gets ignored. Hardcoded to 'source' to make merging both kolla and kolla-ansible changes easier. Change-Id: I6890a7ec7baf507f8acd75b42bbd0fb17c2e4511 Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/837620
2022-04-08kolla: drop 'binary' install typeMarcin Juszkiewicz1-8/+2
We no longer support binary images. This patch removes a way to build them. "--install-type" argument has one available option now. Later in cycle it will be ignored. Change-Id: If0e881d253d6a4c89ffe27831a661df321b6d5b6
2022-03-16Add enabled flag for sourcesMark Goddard1-11/+42
Sometimes it might be useful to disable a source for an image, e.g. for a plugin that is not used. This may be to reduce image sizes, or restrict the dependencies necessary to build images. This change adds a new 'enabled' option to each source definition. This allows sources, plugins or additions for each source image to be disabled individually. Change-Id: I31ee0744574fb3db96bee2e9f545600de53d9ee4
2021-09-03ceph: Update CentOS packages to PacificMark Goddard1-2/+2
Updates the CentOS images to use centos-release-ceph-pacific. This is in line with Ubuntu which gets Pacific from UCA. Debian remains on Nautilus. Change-Id: Ie1e77f9c7d83538c051362a8972745f184c72019
2021-08-24Debian/Ubuntu: handle APT keys in proper wayMarcin Juszkiewicz1-7/+33
apt-key is deprecated and in Debian 'bullseye' it complain when we use it the way we did. The right way is to store key files in separate directory and then, for each repo source file, point to proper key. And this is what this patch does. Both distros use the same keys now. More info: https://michael-prokop.at/blog/2021/02/16/how-to-properly-use-3rd-party-debian-repository-signing-keys-with-apt/ Change-Id: Ie19bf46236907d79cb42a5e94cd1067e952b8cbb
2021-07-28drop leftovers of RHEL supportMarcin Juszkiewicz2-60/+3
As a project we never built RHEL based images so support for them was not tested. Change-Id: I27dfe34ade088228d71f6857ec4f3ab28ee63915
2021-06-10Change network_mode default to hostMichał Nasiadka1-10/+10
Kolla-Ansible deploys Docker without bridge by default [1]. [1]: https://review.opendev.org/c/openstack/kolla-ansible/+/751795 Change-Id: I7430b72f5d8abe59a7151df73e6d59898d942636
2021-05-17add openstack-python3-xena-jobs-arm64 jobricolin1-0/+5
This is a non-voting job to validate py3 unittests on AArch64 Also fixed test units to use source install type with Debian as Kolla does not support Debian/binary on !x86-64 Co-authored-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> Task: 41911 Story: 2007938 Change-Id: I6889555820adcaee8a86e70b5c3ea31c15e3192c
2021-04-26move to Debian 'bullseye'Marcin Juszkiewicz1-1/+1
Bullseye is now in hard freeze cycle and goes for release. https://release.debian.org/bullseye/freeze_policy.html Co-Authored-By: Michal Nasiadka <mnasiadka@gmail.com> Change-Id: I543965a2741cebfa759576a4c75669a7bacd4208
2021-03-05Upgrade from ELK6 to ELK7 FOSS releaseDoug Szumski1-2/+2
Upgrade to the latest ELK OSS release. Due to the licensing change, this effectively pins us to 7.10.x which is supported until 2022-05-11 [1]. This patch also takes advantage of the ARM artifacts which have been kindly provided since the 7.8.0 release [2]. [1] https://www.elastic.co/support/eol [2] https://www.elastic.co/blog/elasticsearch-on-arm Change-Id: Icfa3db5788b25f70ee75411dbaf20d8d4a6a734b
2021-01-20implement "allowed to fail" imagesMarcin Juszkiewicz1-6/+8
Sometimes we have images failing. Not always we care about them. So let allow them to fail and stop caring do they build or not. Change-Id: I01cf3635814255f682d6bd161cf1d985d92a9a31
2020-10-27build: unbreak JSON serializer on failed buildsMarcin Juszkiewicz1-4/+4
Commit 3fa5a236b4d8a31a57e14cbb482e664d5cc0111c converted status consts into Enum. But not completely - this fix gets it done properly. Also fixed tests. Change-Id: Ifdacf62cd4b99c452bd09de7c6cde6fbee75560d
2020-09-30convert STATUS_* consts into EnumMarcin Juszkiewicz1-33/+33
We had that NOTE for quite a while so let's get rid of it. Change-Id: I5d2b29ca38eb1999657435b1eea80b2de1ebeffa
2020-08-07ubuntu: move to 20.04 FocalMarcin Juszkiewicz1-1/+1
There is a time once every 2 years when ubuntu team releases new LTS release. And then UCA joins with binary packages for current OpenStack development cycle. It is this time for Ubuntu 20.04 'focal'. Depends-On: https://review.opendev.org/745156 Change-Id: I045aa6b4b4fd83fbe7d1fda89549f0ef1e88ec12
2020-06-18Remove deployment of FWaaSAlfredo Moralejo1-5/+0
FWaaS is being retired upstream [1][2] in V cycle. [1] https://review.opendev.org/#/c/735838/ Change-Id: I45679c8596844780565644e52034bd3de0915ff4
2020-04-15Merge "Remove support for CentOS 7"Zuul2-37/+10
2020-04-15Remove support for CentOS 7Marcin Juszkiewicz2-37/+10
With the move to RHEL/CentOS 8 we no longer have Python 2 in our images so there is no need for checking which Python version (2.x or 3.x) is used inside of containers. We also no longer have to support yum as a value for distro_package_manager. Partially-Implements: blueprint centos-rhel-8 Change-Id: Ie45cf3465fedddbde7856961527421883ba3d5c9
2020-04-09Upgrade Elasticsearch and Kibana to 6.xSylvain Baubeau1-2/+2
Start using oss tagged binaries. These binaries contain only features that are available under the Apache 2.0 license. Implements: blueprint elasticsearch-kibana-version-upgrade Co-authored-by: Dincer Celik <hello@dincercelik.com> Change-Id: I8b52b2630a72393bf4b9c7095fc67db6c36c27fa
2020-04-04Merge "Fix --skip-existing and --skip-parents"Zuul1-1/+71
2020-04-02Use unittest.mock instead of PyPI mockRadosław Piliszek2-2/+2
Now that py2 is gone, oslotest dropped dependency on mock and will soon affect Ussuri CI [1], let's use unittest.mock built in py3. This also fixes py38 jobs and proactively prevents py36 and py37 failing due to [1]. This is because we never included mock in test-requirements (but in lower-constraints where it does not really belong at all) and instead relied on oslotest to bring it in. [1] https://review.opendev.org/716322 Change-Id: Iceedcc7ad0d087414f6f94dfc0235fda4754be63
2020-03-16Fix --skip-existing and --skip-parentsMark Goddard1-1/+71
There are currently a few corner cases with the --skip-existing and --skip-parents flags. Examples: 1. kolla-build --type source --skip-parents monasca-grafana This does not try to build the monasca-grafana image. This appears to be an issue when the image is a grandchild. 2. kolla-build --type source --skip-parents This will build all images. Expect it to skip parents. 3. kolla-build --type source --skip-existing This will build all images. Expect it to skip existing images. The filter_images method does quite a lot, including handling whether images are buildable, regex/profile matching, and which images to skip. The matching and skipping parts are done in a single pass, which can lead to some weird effects due to dependencies between the images and their statuses. Also, skipping is only currently applied when there is a regex/profile filter. This change splits out the matching and skipping into two separate passes. In the first pass, we mark all buildable images that match the filter as matched. In the second, we iterate over matched images, applying the skip existing and skip parents rules. Change-Id: I2f895aea0cc59d808129e9fc636af0890196af33 Closes-Bug: #1867614 Related-Bug: #1810979
2020-02-26Throw TypeError in repository enable/disable macrosMark Goddard1-0/+13
Currently we do not validate the type of the reponames argument to the enable_extra_repos and disable_extra_repos macros. It should be a list, but we merged a call where the argument was a string, which interpreted as a list of single characters ('e', 'p', 'e', 'l'), and silently ignored. This change validates that the reponames argument is a list to avoid this issue. Change-Id: I497c8b2c1dc548e678698474d3604664cf8eeb07
2020-01-02centos: disable EPEL by defaultMarcin Juszkiewicz1-7/+49
Our images use 'scsi-target-utils' from EPEL and then can be built without it. So let's disable this repository right after installing package from it. Images can enable EPEL with 'enable_extra_repos' macro. Also added 'disable_extra_repos' macro as we have some images in a need. Change-Id: Id70474e7602b531b63fdc11ae690fd9ed33def74
2019-12-17add more tests for enable_repos template methodMarcin Juszkiewicz1-1/+76
- CentOS single, multiple and one with missing repo - Debian single, multiple and one with missing repo - TripleO Change-Id: Ide4868d031bd84b1a6935aada7d0e7627f11bf37
2019-11-29Enable repos only when neededMarcin Juszkiewicz1-0/+12
Disable external repositories by default and enable only when needed. Depends-on: https://review.opendev.org/696480 Implements: blueprint repos-off-by-default Change-Id: Icf2a8397a8349e0fe849d88d160409fd234480a9
2019-10-16handle push error properlyMarcin Juszkiewicz1-0/+40
If there was some error during pushing then Kolla was greeting with "all went fine" message anyway: INFO:kolla.common.utils.aodh-api:Trying to push the image ERROR:kolla.common.utils.aodh-api:Get http://10.101.16.1:5000/v2/: dial tcp 10.101.16.1:5000: connect: no route to host INFO:kolla.common.utils.aodh-api:Pushed successfully This patch changes that. Now if there is an error during push then proper exception is raised to PushTask and image is marked with PUSH_ERROR status. This way at the end of build it is easy to spot which images did not got pushed to registry: INFO:kolla.common.utils:=========================== INFO:kolla.common.utils:Images that failed to build INFO:kolla.common.utils:=========================== ERROR:kolla.common.utils:base Failed with status: push_error ERROR:kolla.common.utils:nova-api Failed with status: push_error ERROR:kolla.common.utils:nova-base Failed with status: push_error ERROR:kolla.common.utils:nova-compute Failed with status: push_error ERROR:kolla.common.utils:nova-compute-ironic Failed with status: push_error ERROR:kolla.common.utils:nova-conductor Failed with status: push_error Closes-Bug: #1848019 Change-Id: Id2ab97bf4c0dc3423268a0ea435b56f4a65f7196
2019-09-19Fix retries when pushing imagesMark Goddard1-0/+34
Currently the retry mechanism is broken for pushing because the image state gets set to an error state, and is never reset to 'built'. This prevents the PushTask from setting success to True. This change sets the image state to 'built' if the push succeeds, ensuring it overrides any previous failures. Change-Id: I93fc0e383da8fec6b3ca31f8094321c2a0c3af71 Closes-Bug: #1844697
2019-09-18Remove support for OracleLinuxMark Goddard1-1/+1
We have agreed to remove support for Oracle Linux. http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006896.html Change-Id: I961dfba1aeef2c509ecdaf376dfe6928926ffb39 Depends-On: https://review.opendev.org/682858
2019-07-21Change how build process treats unbuildable imagesMarcin Juszkiewicz1-4/+11
We have a list of images which are unbuildable for distro/buildtype/arch combos. This patch renames it from SKIPPED_IMAGES to UNBUILDABLE_IMAGES because 'skipped' images is something else in our code. Now all unbuildable images are marked as such and so are their children. At the end of build there is information provided about skipped images (due to --skip-parents or --skip-existing options) and then information about not buildable images. Both list should only contain entries related to requested build. Change-Id: If9b521339f564e483cba03d52e7c4eba271821a5
2019-06-06Debian/Ubuntu: move to Python 3 for source imagesMarcin Juszkiewicz1-2/+2
Commit 43b74ccc157c0b50138c3785ed91dab504895571 enabled use of Python 3 based packages but not switched to use Python 3. Some of images still contain Python 2. There are two reasons: - Ceph (ceph-common depends on Py2) - python3-ldappool on Ubuntu 18.04 In Ceph situation Py3 packages were added. For second one we can not do anything - Py2 dependency got dropped in Ubuntu 18.10 version. Removed neutron-server-plugin-networking-infoblox due to being not maintained. Once https://review.opendev.org/#/c/657578/ get merged someone may revert that part. Implements: blueprint debian-ubuntu-python3 Depends-on: Ie2a1077f7def0743f1403341985e2109aa490026 Change-Id: Ibfe0c2b8be98db56c61f74fb0247488ab3749ef4
2019-04-24Move to opendevZhongShengping2-3/+3
1.Use opendev.org instead of git.openstack.org. 2.Use review.opendev.org instead of review.openstack.org. You can see the discussion below: http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html Change-Id: I377ea63827b6f3995ee275fdc466c1b2100eafc7
2019-03-12Add distro_package_manager configurationAlex Schultz1-0/+46
Add configuration to allow the switch between dnf/yum or provide an alternative package manager. Change-Id: I913769fcf5875a12b4561958cf7c90b74cd116ea
2019-03-12replace 'distro_python3' variable with 'distro_python_version'Marcin Juszkiewicz1-10/+30
RHEL7 derived distros use Python 2.7 RHEL8 uses Python 3.6 Ubuntu 18.04 uses Python 3.6, but we haven't enabled it yet. Debian:buster uses Python 3.7, but we haven't enabled it yet. We have several places in code where "/lib/pythonX.Y/site-packages" is used so we need to know value of X.Y rather than py2/py3 switch. How to check for Python 2: {% if distro_python_version == '2.7' %} Breaks: Iad6b70b433a0dd1b0f8ae6790fd280594517661a Change-Id: Ie24f109af19ae0438c21e0d43cb3cc947812b4ce
2019-01-16Add base_package_typeAlex Schultz1-0/+18
Currently in all the Dockerfile.j2 files, we are doing base distro if conditions for package names when installing via a binary. Rather than duplicating these base_distro in [<distros>], let's provide a single value for determining the binary installation method. This change adds a base_package_type which we can use to lookup if 'rpm' or if 'deb' rather than relying on specific distro names. Change-Id: I8a0c6c8fbc32a8cfa6932fddf28a449fceda3d49
2019-01-16Add distro_python3 optionAlex Schultz1-0/+20
As we add support for python3 on RHEL based systems, we need to add a flag to key off of in the Dockerfiles to swap out the python3 package names. This change adds a distro_python3 configuration option that can be defined to force python3 or python2 package names. If this option is not provided, we set this to True for RHEL 8 based distros and false for other distros. Change-Id: I4028991bad92c0e8e21066cc4173c06ce5eba393 Related-Blueprint: python3-support
2018-06-30Ability to build container against docker swarmVictor Coutellier1-5/+5
Add a decode=True option during call to dockerAPI in order to build container against recent docker swarm. Closes-Bug: #1779460 Change-Id: I48f041e7ed9dd5a5f410b4eaa081f6f013119d17
2018-05-16Fix image push for python3Michal Nasiadka1-2/+3
Rework push_image function to use docker client push function with decode=True, to decode automatically to dict. Change-Id: I28d38e968c3809a1074347271e49dc07dc861f11 Closes-Bug: 1771514
2018-04-20Add network_mode for image buildingAlex Schultz1-4/+21
Add the ability to specify a different network_mode when running the image builds. Under the covers the default is to use bridge networking but if that is unavailable or a user would like to specify a different networking method they can now do so using the [DEFAULT]\network_mode configuration. Change-Id: I1d2398273b57ca4d52bbfacbc896ffecc58bc54b
2018-02-03Remove the unsupported 'insecure_registry' for >=docker 3.0.0Jianghua Wang1-1/+12
The argument of 'insecure_registry' for image push has been removed by this commit (in docker 3.0.0): https://github.com/docker/docker-py/commit/b180b8770a265e33099bd6da76c3e556a1028491 Actually this argument was marked as deprecated and unfunctional before this removing. This commit is to remove this parameter at invoking the push() function when docker version >= 3.0.0. Change-Id: Ifa7d304d08a4073dcb1dff751d5a443c112cd377 Closes-Bug: #1746703
2017-08-03Fix for git clone issue during kolla image buildJiri Prokes1-0/+22
When the first build attempt fail for any reason (e.g. short network outage, proxy outage,...) then the second attempt always fail for git source type. Git can't clone repository because destination directory exists and is not empty. Current fix checks destination directory and if it exists then it is removed before cloning. Change-Id: I949140c49a64baea579d61047e3b2f1240da2771 Closes-Bug: #1706369
generated by cgit v1.2.3 (git 2.39.1) at 2026-11-05 18:23:14 +0000