aboutsummaryrefslogtreecommitdiff
path: root/kolla/image/build.py
AgeCommit message (Collapse)AuthorFilesLines
2026-06-18kolla-build: Add Docker BuildKit support via docker buildx buildMichal Nasiadka1-0/+10
Add --buildkit flag to use docker buildx build instead of the docker-py SDK. An optional --buildkit-builder selects a named buildx builder instance, enabling docker-container or remote drivers for multi-arch builds. --buildkit and --squash are mutually exclusive and rejected at startup. PushTask is skipped when buildkit is active; push is handled inline. Set buildkit as the new default. Change-Id: Idbf1d1bdc091bb84721f8006c2b916731b999859 Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
2024-11-15Add mechanism for patching files in containersMichal Arbet1-0/+1
This patch adds a way to patch files in a Docker image built by Kolla. This is very useful for several reasons, specifically: - Custom modifications - The stable branch of some library has a fix but no pip package has been released - Eliminates the need to package your own pip packages - Eliminates the need to invent your own versioning to prevent upstream versioning - Eliminates the need to manage a pip server - In other words, it eliminates the need to get a wheel into the image and install it manually using any method not previously mentioned It is also highly desirable because, although Kolla can replace the source for a service with a custom URL for a tarball or its own Git repo, it cannot do this for dependencies pulled from pip. I would also like to point out that this is a feature with its own code path and works only if the user "inserts" a patch into the folder patches/docker-image/something.patch and creates an analogous series file for patch source code. Simply said, this code will never interfere with the upstream build process since this feature is not intended for use in upstream. It is rather meant for downstream users who know what they are doing. Now they just have an option to patch their images. Everything works on all layers of the Docker image and stores a report of applied patches which can then be seen in /etc. This mechanism is similar as debian patch quilt. Change-Id: I61d0790c5d4d070b7ea9e8c99c0a76ff5d22bf9d
2024-05-28Drop packaging requirement and unpin docker-pyMichal Nasiadka1-11/+9
Change-Id: I11d306be5e5bb3d5b05b50dd8ee64ac9424854a1
2023-12-21Merge "Summary JSON file"Zuul1-4/+1
2023-11-28import packaging only for docker container engineBartosz Bezak1-1/+1
Change-Id: If4e0c0e5d6ed5c79ca7d8a3c8f16bfa665f6968f
2023-09-21Add podman for image buildingMichal Arbet1-2/+12
Add podman as an option to choose for container engine for kolla-build (--engine podman) Signed-off-by: Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Co-Authored-By: Michal Arbet <michal.arbet@ultimum.io> Change-Id: I068c906df97745e397408d8c3ef6af47ee037638
2023-09-08build: distutils.StrictVersion() is deprecatedMarcin Juszkiewicz1-2/+2
kolla/tests/test_build.py::MainTest::test_run_build [65/1834] /home/marcin/devel/linaro/kolla/kolla/kolla/image/build.py:117: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. StrictVersion(docker.__version__) Change-Id: I2ebc8b0e35c67bf893deb89ea7fa0146dedd4640
2023-05-05Fix docker version check after docker-py>=6.0.0Michal Nasiadka1-1/+2
docker-py reworked their version function and now it does not return a string [1]. Example: >>> import docker >>> docker.version <module 'docker.version' from '/home/cloud-user/kolla-venv/lib64/python3.9/site-packages/docker/version.py'> [1]: https://github.com/docker/docker-py/commit/cd2c35a9b699522b282cc4f024efa5699df24896 Closes-Bug: #2018559 Change-Id: I5ec0fbfdaf18cf53a6b5bf91236d3e040daa9d7e
2023-04-27Summary JSON fileMark Goddard1-4/+1
Add support for writing the build summary to a JSON file. This is more machine-friendly than the current approach of writing to standard output at the end of the build. The file is configured via [DEFAULT] summary_json_file. Change-Id: I9868af5d00977750d0673fc48f8863a99f878dc8
2023-02-27Merge "Add a way to add other container engines"Zuul1-5/+24
2023-01-30use underscore for unused variable in loopHoai-Thu Vuong1-2/+2
Change-Id: If42a89642f0322c52bed44f402a8dcbb9c0a9bd6
2022-11-08Add a way to add other container enginesMarcin Juszkiewicz1-5/+24
Add engine adapter module to allow additon of other container engines, create one common EngineClient that would be called in all other modules. Based on patch by Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Change-Id: Ice6467086bd292af086322afc3fc4e869d89eefa
2022-10-07Drop executable perms from non-executablesRadosław Piliszek1-0/+0
It does not make much sense to have them there. TrivialFix Change-Id: If66d264dc8cc366c9f51bfe36f96dd64d4c68d9e
2022-10-07Merge "do not play with PROJECT_ROOT in build.py"Zuul1-20/+5
2022-10-06do not play with PROJECT_ROOT in build.pyMarcin Juszkiewicz1-20/+5
PROJECT_ROOT mangling is already done in kolla/cmd/build.py (which is called as 'kolla-build' or 'tools/build.py' or directly) so it is not needed here. Now we can sort imports and drop not needed ones. Change-Id: Iac6ea537491e5c00ce6fb4b4e8f07a9124e9fa8b
2022-10-06drop shebang from Python filesMarcin Juszkiewicz1-2/+0
Only kolla/cmd/build.py is an executable script used by user (either as 'kolla-build' or 'tools/build.py' or directly). Change-Id: Ia1ba2699f425749f9d4f2cdba27caeb97bc3d668
2022-10-06Move KollaWorker stuff to separate fileMarcin Juszkiewicz1-733/+1
As part of kolla/image/ refactoring we move KollaWorker related classes into own file. Based on patch by Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Change-Id: I55c3f6b34631048b2463b02e6ea611d7e4c1a912
2022-10-06Move build related tasks to separate fileMarcin Juszkiewicz1-376/+1
As part of kolla/image/ refactoring we move build related task classes into own file. Based on patch by Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Change-Id: I093620679016b37e1664c9fe4cf7559433e744b7
2022-10-06Move status and log related things to separate fileMarcin Juszkiewicz1-22/+3
As part of kolla/image/ refactoring we move status enums/consts into own file. Based on patch by Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Change-Id: I940816c7e487f6d5c7abc4446226e2992cf9e63d
2022-10-06Move info about unbuildable images to separate fileMarcin Juszkiewicz1-56/+1
As part of kolla/image/ refactoring we move information about unbuildable images to own file. Like we did with users and sources in past. Based on patch by Konstantin Yarovoy <konstantin.yarovoy@tietoevry.com> Change-Id: I759ab49a3ef488042984067c1eca93e945805678
2022-10-03proxysql: Use almalinux for centos/rocky9Michal Nasiadka1-2/+0
As per https://github.com/sysown/proxysql/issues/3989 Change-Id: If077d60e9efd29fab80f4f5188a4f54015734a92
2022-09-30influxdb: enable on CentOS/AArch64Marcin Juszkiewicz1-1/+0
Change-Id: I468e68db23c6785bd3bfbe65733fe5bf52295fbb
2022-09-30Add RockyLinux 9 supportMichal Nasiadka1-1/+12
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/859570 Change-Id: I7af51d07b8a939c908aa072294061def57dd67de
2022-09-27grafana: ignore SHA1 gpg key on CentOS Stream 9 (and family)Marcin Juszkiewicz1-1/+0
RHEL 9 refuses to use SHA1 gpg keys. Grafana team still did not handled key upgrade: https://github.com/grafana/grafana/issues/41036 Change-Id: Id963271103ce1397b7a5498d032e8f797edfcc70
2022-09-27Move to CentOS Stream 9Marcin Juszkiewicz1-2/+10
OpenStack 'zed' requires Python 3.8+ so RHEL 8 family has to go. This changeset moves to CentOS Stream 9 while move to RockyLinux 9 is planned as final solution. CI moved to CentOS Stream 9 nodes. Depends-on: https://review.opendev.org/c/openstack/kolla-ansible/+/839715 Change-Id: I113b9984294cf8663d3fc0c8840320e1d40ea731
2022-09-23Merge "ubuntu: enable monasca for x86-64"Zuul1-2/+1
2022-09-14Python 3.8+ has better shutil.copytree()Marcin Juszkiewicz1-10/+1
doc says: > If dirs_exist_ok is false (the default) and dst already exists, a > FileExistsError is raised. If dirs_exist_ok is true, the copying > operation will continue if it encounters existing directories, and files > within the dst tree will be overwritten by corresponding files from the > src tree. So we do not have to copy dirs/files one by one and tell shutil.copytree() to handle everything. Change-Id: I97d9c8261351b8fd88f088c1463cc508e07792e7
2022-09-13ubuntu: enable monasca for x86-64Marcin Juszkiewicz1-2/+1
There is confluent-kafka wheel on Pypi so we can build those images on x86-64. Change-Id: I16a9656641932114e02f6ffa76399c0c4106a90a
2022-08-22Bump minimum docker-py versionDr. Jens Harbott1-7/+0
With version 6.0.0 of the docker python module, their implementation of versioning has changed, making our check for old versions fail. Since version 3.0.0 is from 2018, we should be safe to assume that as a minimum version though, so we can just get rid of the special handling of older versions. Change-Id: I077b7b5acf2c1f9beb6da06d3555e2ebe30831d1
2022-07-27Move to Ubuntu Jammy (22.04)Michal Nasiadka1-2/+6
mariadb - downloads.mariadb.com doesn't have jammy version, should be there soon - using Ubuntu provided version for now telegraf/collectd - collectd packages missing in Jammy - marked as unbuildable monasca - requires new librdkafka, no wheel for Python 3.10 for now Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/846070 Change-Id: I11b4fbcaba165824252902d0270ce1dbbc3d6e9d
2022-07-01Merge "Change kolla_version LABEL to git sha-1"Zuul1-1/+2
2022-06-21Change kolla_version LABEL to git sha-1Michal Arbet1-1/+2
Downstream users are often building their own custom images from their own forked git. Reasoning can be various - added fixes not merged yet, some customizations and etc. Because of this, it's very usefull to have information about git commit in images, and this is exactly what that patch does. If images are built from pip source, kolla_version LABEL contains kolla version. Change-Id: If81989603185528dae86df52e8f8d0f94cdca85d
2022-06-20disable monasca for AArch64 on all distrosMarcin Juszkiewicz1-4/+2
INFO:kolla.common.utils.monasca-base: Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html" No more monasca for aarch64. Change-Id: Ice6f1ab4a08d0ef07ce2434bfef704b68e90f9d0
2022-06-20Fix error message when git clone failsMark Goddard1-1/+2
1. There was no %s for the image.name 2. The image name is in the logger, so the source is more useful (which may be a plugin or addition) TrivialFix Change-Id: If95a8a41d2caf6738d4ccb9ddb651a06018743f4
2022-06-02Allow to provide own repos.yaml fileMarcin Juszkiewicz1-0/+2
Several people use Kolla behind firewall/proxy. Internal mirrors are often in use then. We do not provide a way to replace repos.yaml file in an easy way which may lead to complicated solutions or template overrides. This patch adds a way to provide own copy of repos.yaml file. Change-Id: I0b07da22fea27e0ff4e90aaad19e50d84ff9a121
2022-05-29Use the new image naming scheme by defaultRadosław Piliszek1-4/+1
Adds a relevant reno to inform users about the change and the possibility to customise the prefix. Additionally, this patch includes the fix to Ubuntu image builds regarding rabbitmq and erlang. Both the changes are required to fix master CI. Change-Id: I6bc8b489a810849744c8bead6b9e350c5d3e36b7 Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/843752
2022-05-18Fix local sources of git repositoriesRadosław Piliszek1-10/+11
This is I2cbf1f539880d512aa223c3ef3a4b19ee18854ac extended to fix the case when a git repository is used with a git repo. This is probably a rarer use case but, still, we use it in CI for in-review changes testing. Change-Id: I77b0dcd2e9dfd8ea8390a471b80c8954b67ef91b
2022-05-18[bifrost] Force Bifrost to use the correct git branchRadosław Piliszek1-0/+5
Bifrost decides on the branches to use by itself. Early in the cycle, they are often not set correctly and may break our CI. This patch ensures we force Bifrost to use the expected branches, making us independent of Bifrost getting fixed. Change-Id: I33dbca927062284fd7a06ab6db732dd5e2219c40
2022-05-02[publishing] use a new naming schemeRadosław Piliszek1-1/+4
This change switches published images names from the current format <base_distro>-<image_name>:<openstack_release>-<arch_suffix> to a preciser and image-oriented (for better variant listing) format <image_name>:<openstack_release>-<base_distro>-<base_distro_version>-<arch_suffix> Change-Id: I6a4b5c2087b94f96f72aa9d0d0781ff14d3ee6bb
2022-04-21switch to short image names by defaultMarcin Juszkiewicz1-4/+1
We can drop temporary 'use-short-names' argument and make short (kolla/debian-nova-libvirt:master) image names default. Kolla Ansible is using short names now. Change-Id: I153355353cd12aa25adaa9c87438a478748dfb28
2022-04-19add a way to not have 'install_type' in image name (temporary)Marcin Juszkiewicz1-2/+4
--use-short-names argument drops 'install_type' from image names: kolla/debian-openstack-base:14.1.0 This should allow to get rid of '-source' in image names without disrupting CI jobs. Switch publish jobs to use short names so k-a tests will have ready to use images for both long and short names. Change-Id: I29c9d51f4b518a4e2d3157ab5d6cc434fb83f2ef
2022-04-14drop install_type almost everywhereMarcin Juszkiewicz1-65/+60
The only place where it stays is argument parsing where it gets ignored. Hardcoded to 'source' to make merging both kolla and kolla-ansible changes easier. Change-Id: I6890a7ec7baf507f8acd75b42bbd0fb17c2e4511 Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/837620
2022-04-14Fix image builds with sources using a type=gitMarcin Juszkiewicz1-1/+11
A recent change to git [1] introduced a new behaviour to work around a CVE [2] that disallows any git operations in directories not owned by the current user. This may seem unrelated to installation, but it plays havoc with PBR, which calls out to git to get to get revision history. So if you are "pip install"-ing from a source tree you don't own, the PBR git calls in that tree now fail and the install blows up. When using type=source, kolla clones the repository, then creates a tarball from it, which is ADDed to the image. The ownership of the files in the tarball is preserved, which in this case will be the user running kolla-build. Since the Docker build runs as root, we hit the PBR issue. Our solution is to make sure that any tarball we generate from git sources have all files owned by root:root so that the root user is able to use git commands when building container images. [1] https://github.com/git/git/commit/8959555cee7ec045958f9b6dd62e541affb7e7d9 [2] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24765. Closes-Bug: #1969096 Related-Bug: #1968877 Co-Authored-By: Mark Goddard <mark@stackhpc.com> Change-Id: I2cbf1f539880d512aa223c3ef3a4b19ee18854ac
2022-04-12Merge "drop list of images unbuildable for source type"Zuul1-3/+0
2022-04-11Remove duplicate entry in UNBUILDABLE_IMAGESRadosław Piliszek1-1/+0
hacluster-pcs is already in centos, no need to be in centos+aarch64. TrivialFix Change-Id: I9868f73172efedf113c3cc1567098b3439e2a205
2022-04-11Merge "for Zed cycle minimal Python version is 3.8"Zuul1-1/+1
2022-04-11drop list of images unbuildable for source typeMarcin Juszkiewicz1-3/+0
If they are unbuildable then they should be in distro or distro+arch blocks now. Change-Id: I9cd225d3a7fd99fdeaf4926730c95d37f8f7eac7
2022-04-11for Zed cycle minimal Python version is 3.8Marcin Juszkiewicz1-1/+1
When everything fails assume we have Python 3.8 (should never happen). Change-Id: I69033cc910dd8236a470fd130816f2870a9b5b3a
2022-04-11Drop qdrouterd imageMarcin Juszkiewicz1-2/+0
It was marked deprecated in Yoga. Change-Id: Iea1cf9c539a73da625e335d6c1adb6cd71ec93d4
2022-04-11Merge "drop infra_image_prefix"Zuul1-4/+1
generated by cgit v1.2.3 (git 2.39.1) at 2026-11-05 21:05:19 +0000