aboutsummaryrefslogtreecommitdiff
path: root/kolla/image/build.py
AgeCommit message (Collapse)AuthorFilesLines
2019-09-19ppc64le: add UNBUILDABLE_IMAGES entriesMarcin Juszkiewicz1-1/+35
Change-Id: I6b8f52a1f2227ecb74e3ae59ef1e67d5b5ce1886
2019-09-18Merge "build: enable Octavia for Ubuntu/binary"Zuul1-1/+0
2019-09-18Remove support for OracleLinuxMark Goddard1-8/+1
We have agreed to remove support for Oracle Linux. http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006896.html Change-Id: I961dfba1aeef2c509ecdaf376dfe6928926ffb39 Depends-On: https://review.opendev.org/682858
2019-09-18build: enable Octavia for Ubuntu/binaryMarcin Juszkiewicz1-1/+0
Change-Id: Icc68b404d8713a9fcf0026374378d1645fe30d52
2019-09-17build.py: fix pep8 warningsMarcin Juszkiewicz1-6/+6
Change-Id: Ifaeed6917636e2bcb5ea3bda32ae0393ebb08a4f
2019-09-12Drop Percona XtraBackup in favour of MariaDB BackupMarcin Juszkiewicz1-2/+0
XtraBackup started failing to install on Ubuntu and does not support MariaDB 10.3 version. Related-Bug: #1843043 Change-Id: I9cf62c8e03a02574d85e5349df1d23a51d4c99be
2019-08-31disable 'hacluster-pcs' on aarch64 due to lack of packageMarcin Juszkiewicz1-0/+1
Change-Id: I3000136fa883f510b24133907a66efd194b40266
2019-08-19debian: add some images to unbuildable listMarcin Juszkiewicz1-1/+6
We do not have package repositories for opendaylight, telegraf, xtrabackup images. bifrost tries to install 'mysql-server' package which is not present in Debian 'buster' monasca-grafana has some SSL related issues Change-Id: I795771569a3dcb9730063bd951c230806a08b3c0
2019-08-08Bump up pkg versions in kolla-toolboxMichal Nasiadka1-0/+2
Currently we use Ansible 2.2 in kolla_toolbox, while lowest supported version is 2.6 - bumping up to 2.8. * Use upper constrains in Dockerfile.j2, instead of pinning them in here - Ansible is pinned to 2.8.* * Add no_log=True in kolla_keystone_user * Add endpoint_type as a param in kolla_keystone_service - now AnsibleModule requires all options to be defined, otherwise it throws an UnsupportedParameter error * Add openstack_release variable in kolla/common/config.py which is used for fetching upper-constraints.txt for desired release Change-Id: I2cb0ea2fde1b408d48b39bf7a54b898e850d31aa
2019-07-22Merge "Change how build process treats unbuildable images"Zuul1-36/+91
2019-07-22Merge "Allow building images if parent was skipped"Zuul1-1/+3
2019-07-21Change how build process treats unbuildable imagesMarcin Juszkiewicz1-36/+91
We have a list of images which are unbuildable for distro/buildtype/arch combos. This patch renames it from SKIPPED_IMAGES to UNBUILDABLE_IMAGES because 'skipped' images is something else in our code. Now all unbuildable images are marked as such and so are their children. At the end of build there is information provided about skipped images (due to --skip-parents or --skip-existing options) and then information about not buildable images. Both list should only contain entries related to requested build. Change-Id: If9b521339f564e483cba03d52e7c4eba271821a5
2019-07-17add some more not buildable imagesMarcin Juszkiewicz1-10/+5
Cyborg-agent binaries are only provided as x86-64 rpm packages. CentOS/aarch64 does not have 'mongodb' in EPEL. Ubuntu/aarch64 lacks 'sensu' package. Change-Id: I0fd668018b37e1d64782fd32ff3846a688c4e915
2019-07-11Add Masakari containersKien Nguyen1-0/+1
Masakari provides Instances High Availability Service for OpenStack clouds by automatically recovering failed Instances. Change-Id: I279d6b43f8e4d4e30b39deb607823ceca272a4b8 Implements: blueprint masakari-support
2019-07-03Merge "build: add timestamps to start/end of build"Zuul1-2/+5
2019-06-28Add more entries into SKIPPED_IMAGESMarcin Juszkiewicz1-0/+35
Did build of all images for AArch64, looked at failures and here is a list of ones which never build due to lack of packages. There are some images which could get to the list but there are already patches for them in Gerrit. Also added three more entries for Debian due to missing packages. Change-Id: Ie259a38aad84bd8f5392f19791edfbd1f8ec4b1a
2019-06-26build: add timestamps to start/end of buildMarcin Juszkiewicz1-2/+5
During recent CI issues I got an idea that it would be nice to have information how much time building specific image took. On local systems we can do log and then check 'modified_time - creation_time' but not on CI jobs. Fresh image: INFO:kolla.common.utils.base:Building started at 2019-06-26 13:53:08.393544 INFO:kolla.common.utils.base:Ended at 2019-06-26 13:56:40.354761 (took 0:03:31.961225) Rebuild: INFO:kolla.common.utils.base:Building started at 2019-06-26 14:00:34.913194 INFO:kolla.common.utils.base:Built at 2019-06-26 14:00:38.988874 (took 0:00:04.075686) Failed images do not get second line due to logger.exception() being used. Change-Id: Id13545224fa0fbb729d46386a10505e0989fc3c6
2019-06-19Make SKIPPED_IMAGES more flexibleRadosław Piliszek1-80/+43
With introduction of new architectures, it seems reasonable to allow skipping of images based on their architecture. Additionally, the existing framework had unneccessary duplication. This commit makes the feature more flexible and cleaner. Existing skips have been programmatically rewritten. Change-Id: I34c002a0ee605ca121d9d923b256e4fe1ea3d9aa Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-06-18Merge "Remove crane image"Zuul1-5/+0
2019-06-07Remove crane imageMark Goddard1-5/+0
This image was deprecated in the Stein cycle. Change-Id: I249b78cd263c97c4a9223523c77f8c325f3e649a
2019-06-07Merge "Debian/Ubuntu: move to Python 3 for source images"Zuul1-0/+4
2019-06-06Debian/Ubuntu: move to Python 3 for source imagesMarcin Juszkiewicz1-0/+4
Commit 43b74ccc157c0b50138c3785ed91dab504895571 enabled use of Python 3 based packages but not switched to use Python 3. Some of images still contain Python 2. There are two reasons: - Ceph (ceph-common depends on Py2) - python3-ldappool on Ubuntu 18.04 In Ceph situation Py3 packages were added. For second one we can not do anything - Py2 dependency got dropped in Ubuntu 18.10 version. Removed neutron-server-plugin-networking-infoblox due to being not maintained. Once https://review.opendev.org/#/c/657578/ get merged someone may revert that part. Implements: blueprint debian-ubuntu-python3 Depends-on: Ie2a1077f7def0743f1403341985e2109aa490026 Change-Id: Ibfe0c2b8be98db56c61f74fb0247488ab3749ef4
2019-06-05Allow building images if parent was skippedJason1-1/+3
This fixes a bug where if an image had a parent that was skipped, it would never get built. Normally when a parent finishes building it triggers the build of its children. This process is kicked off by Kolla finding the grandparents that have no parents above them. However, sometimes parents should be skipped, and in this case they would never get scheduled, and so their children would never build, producing a strange error without much information. Fix is to allow building an image if its parent is explicitly marked as having been skipped. The use-case for this is when you want to iterate on building an image without going through the cost of re-running the parent builds. In particular, if your builds are using Git sources, which break Docker caches, the cost of parent builds is high and the feedback loop during development has a much higher latency. Allowing some way for images to be rebuilt in a targeted manner is helpful when in the development phase. Change-Id: I6b6897d20626825bcd235b9a85859644effbd222 Closes-Bug: #1810979
2019-06-05Remove tripleo-uiAlex Schultz1-12/+0
The tripleo-ui project has been removed from the upstream. Remove the references to the tripleo-ui containers. Change-Id: I353ecd3ed04c78fd21f0108c5e1355280d3f3bb9 Closes-Bug: #1831478
2019-05-20Add Qinling containersGaetan Trellu1-0/+4
Add Qinling Dockerfile , Qinling is an OpenStack project to provide "Function as a service". This project aims to provide a platform to support serverless functions. Change-Id: Idebf5c8756a0d0225adcce29b45a399120bb5eb6 Implements: blueprint qinling-support
2019-04-03Add config option use_dumb_init to add it to container imagesRabi Mishra1-0/+2
Usage of dumb-init can also be disabled from cli with `--nouse-dumb-init` when building images. Also moves ENTRYPOINT outside of dumb_init_installation block. Change-Id: Id32d54fc9913b83cf121ede5a5e265e0772062ae Related-Bug: #1821970
2019-03-27Add cyborg to kollaBai Yongjun1-0/+6
Because kolla not have cyborg so should add this. Implements: blueprint add-cyborg Change-Id: I497e67e3a754fccfd2ef5a82f13ccfaf890a6fcd
2019-03-21Merge "Introduce clean_package_cache"Zuul1-1/+4
2019-03-18Properly set debian_arch for ppc64el hardwareMichael Turek1-0/+2
On Red Hat distros, ppc64el is referred to as ppc64le. This patch simply sets debian_arch to ppc64el if the base arch comes up as ppc64le. Change-Id: Idb9b30f792db11bea9b7d96025bd20a03fabc78a
2019-03-17Introduce clean_package_cacheEmilien Macchi1-1/+4
Default to true (to maintain backward compatibility), this parameter will allow to disable the Package cache cleaning when building the Kolla images. In TripleO, we are investigating how we can accelerate the builds by not cleaning the Yum cache and this patch will help us to do so. Also included same for deb package_install method. Co-Authored-By: Jon Schlueter <jschluet@redhat.com> Change-Id: I0adc85302dfe0411d362c899f3e0d6a0019680d4
2019-03-12Add distro_package_manager configurationAlex Schultz1-0/+13
Add configuration to allow the switch between dnf/yum or provide an alternative package manager. Change-Id: I913769fcf5875a12b4561958cf7c90b74cd116ea
2019-03-12replace 'distro_python3' variable with 'distro_python_version'Marcin Juszkiewicz1-7/+8
RHEL7 derived distros use Python 2.7 RHEL8 uses Python 3.6 Ubuntu 18.04 uses Python 3.6, but we haven't enabled it yet. Debian:buster uses Python 3.7, but we haven't enabled it yet. We have several places in code where "/lib/pythonX.Y/site-packages" is used so we need to know value of X.Y rather than py2/py3 switch. How to check for Python 2: {% if distro_python_version == '2.7' %} Breaks: Iad6b70b433a0dd1b0f8ae6790fd280594517661a Change-Id: Ie24f109af19ae0438c21e0d43cb3cc947812b4ce
2019-02-28debian/ubuntu: make use of Python3 based packagesMarcin Juszkiewicz1-1/+0
Both Ubuntu Stein UCA and Debian 'buster' migrated their OpenStack packages to Python 3. Note that Debian 'buster' is not released yet and contains Rocky packages. Stein ones will be available later. Co-Authored-By: Lee Yarwood <lyarwood@redhat.com> Co-Authored-By: Eduardo Gonzalez <dabarren@gmail.com> Change-Id: I160f79cc57f54ec3eac857c5babd1a6e2656d228
2019-02-19debian/ubuntu: introduce 'debian_arch' variableMarcin Juszkiewicz1-0/+6
Many images have 'if x86_64 then amd64 elif aarch64 then arm64' check to comply with Debian like architecture names in several projects. This patch creates 'debian_arch' variable which can be used in all Docker files (similar to 'base_arch' one). All required images got converted to use it. Change-Id: I9c5e6f13d6c9b24fe323408512bd5aef290111ad
2019-01-31build: exit if there is no Docker connectionMarcin Juszkiewicz1-1/+3
When user starts Kolla on system without Docker daemon running huge traceback is generated and then exit. Issue is because KollaWorker raises exception but no one catches it (then only call to it is in run_build() which assumes everything is ok). Change-Id: I68ea3f48c75b51dbbab3accbc531d2f73428c48f
2019-01-30Allow running more commands dockerlessMartin André1-1/+4
The `--save-dependency`, `--list-images`, and `--list-dependencies` options to the kolla-build commands do not actually require docker daemon running. This commit stop kolla-build from complaining when it is called with one of the above options without docker daemon started. Change-Id: I966587236aca363316db8abbfc04763655b4b68f
2019-01-16Add base_package_typeAlex Schultz1-0/+9
Currently in all the Dockerfile.j2 files, we are doing base distro if conditions for package names when installing via a binary. Rather than duplicating these base_distro in [<distros>], let's provide a single value for determining the binary installation method. This change adds a base_package_type which we can use to lookup if 'rpm' or if 'deb' rather than relying on specific distro names. Change-Id: I8a0c6c8fbc32a8cfa6932fddf28a449fceda3d49
2019-01-16Add distro_python3 optionAlex Schultz1-0/+9
As we add support for python3 on RHEL based systems, we need to add a flag to key off of in the Dockerfiles to swap out the python3 package names. This change adds a distro_python3 configuration option that can be defined to force python3 or python2 package names. If this option is not provided, we set this to True for RHEL 8 based distros and false for other distros. Change-Id: I4028991bad92c0e8e21066cc4173c06ce5eba393 Related-Blueprint: python3-support
2018-12-27Update hacking versionconfi-surya1-7/+7
Use latest release 1.1.0 and compatible changes w.r.t pep8 Change-Id: Ia3d1166a2746a46591e4e5a04ea91e03ddc28d93
2018-12-19Split placement-api from novaEduardo Gonzalez1-0/+1
This change breaks out the Placement service from Nova ahead of it's removal from the project later in Stein. For the time being the original Nova Placement images are left in place but they will be removed later in the cycle once the Placement code is removed from the Nova project by the following change: https://review.openstack.org/#/c/618215/ Co-Authored-By: Lee Yarwood <lyarwood@redhat.com> Change-Id: I2bf6f5bf45b76e6900f41c21e9af30c69dc6e9c0
2018-10-25build: remove link to failed log before creatingMarcin Juszkiewicz1-3/+9
Kolla allows to create separate logs per image. It also makes very useful symlinks to those builds which fail. But it fails to do that on next builds: ERROR:kolla.common.utils:aodh-base Failed with status: error Traceback (most recent call last): File "./tools/build.py", line 40, in <module> sys.exit(main()) File "./tools/build.py", line 30, in main statuses = build.run_build() File "/home/hrw/devel/kolla/kolla/image/build.py", line 1333, in run_build results = kolla.summary() File "/home/hrw/devel/kolla/kolla/image/build.py", line 1037, in summary "000_FAILED_%s.log" % name)) FileExistsError: [Errno 17] File exists: 'aodh-base.log' -> 'logs/debian-source-buster/000_FAILED_aodh-base.log' Change-Id: I85132bbace63c8b386f69ddb758a3b52e52c0ef0
2018-09-21Add crane docker imageDan Prince1-1/+6
Crane is a lightweight 'pull only' container registry implementation. TripleO is aiming to replace our use of docker-distribution with crane for the Undercloud. Co-Authored-By: Martin André <m.andre@redhat.com> Change-Id: I59d9521f99243401d7703a3dedc26b5baed2d595
2018-09-06Add a tripleoclient containerDan Prince1-0/+6
We'd like to use this container both interactively (for ad-hoc commands) and also to be able to generate Ansible (via the undercloud/all-in-one installers) for TripleO. Partially-implements: https://blueprints.launchpad.net/tripleo/+spec/zero-footprint Change-Id: Ia4d7fdcb9c96d76e191dcc1a46be095b37189b3e
2018-08-03skipped_images None handlingAttila Fazekas1-3/+4
skip the skipped_images loop when the combination is unknown, or when the skiplist is empty. Change-Id: Ia2dfbb1c80baeaa2eeb4720381e4da6c7b1c85af
2018-07-31Add neutron dragent package for RPM binary buildsEduardo Gonzalez1-2/+0
Change-Id: Ief8f075a57f8e365499fddbca25b1af71ccfc96c Closes-Bug: #1772552
2018-07-26build with --template-only to not require running docker serviceJon Schlueter1-1/+6
Add a try/except check and raise excpetion if not --template-only Reason for this is so that you can run kolla_build --template-only on a system with a user that doesn't have permissions to access docker or system may not have docker service running. The template only generation is ok because it doesn't specifically need the docker context Change-Id: I191f3e073d14e9ba5419a8a9b043e8afaa565e2a
2018-07-23Revert "Add networking-odl image to skipped image list"confi-surya1-9/+0
This reverts commit I57594742e0842c0e701b9974046d9d1704446f4b Add ceilometer package into neutron-server-opendaylight container, then it chould be found by pip. Closes-Bug: #1764621 Co-Authored-By: Jeffrey Zhang <zhang.lei.fly@gmail.com> Change-Id: Ibaaf1de0c437ce9d25c93e68ab5a43279c4e60ee
2018-07-16Add support for deploying Monasca ThreshDoug Szumski1-0/+4
Monasca Thresh is a Java topology for Storm which generates alerts from metrics. It depends on Java code in the monasca-common library. In this change we support building the Monasca Thresh Storm topology from source. To enable us to submit the topology to Storm we inherit from the Storm container (rather than the Monasca base image) so that we can use the Storm client. Change-Id: I389af89086cdab6d243c4a4fa65fd157535a5045 Partially-Implements: blueprint monasca-containers
2018-06-30Ability to build container against docker swarmVictor Coutellier1-9/+9
Add a decode=True option during call to dockerAPI in order to build container against recent docker swarm. Closes-Bug: #1779460 Change-Id: I48f041e7ed9dd5a5f410b4eaa081f6f013119d17
2018-06-26kolla-build: Wrap tpl_dict.keys() in a list()Martin Chlumsky1-1/+1
In python 3, tpl_dict.keys() returns a dict_keys object that does not support indexing and is then indexed. Wrap tpl_dict.keys() in a list() so it can be be indexed in python 2 and 3. Change-Id: I17e3bbfc4eeba58e4bdd7fdd1ecdb21baa5c4461 Closes-Bug: 1778781