summaryrefslogtreecommitdiff
path: root/kolla/cmd/build.py
AgeCommit message (Collapse)AuthorFilesLines
2021-01-20implement "allowed to fail" imagesMarcin Juszkiewicz1-2/+2
Sometimes we have images failing. Not always we care about them. So let allow them to fail and stop caring do they build or not. Change-Id: I01cf3635814255f682d6bd161cf1d985d92a9a31
2020-03-16do not use unversioned Python binaryMarcin Juszkiewicz1-1/+1
We said 'bye' to Python 2 already. So let us use Python 3 for all scripts. Unversioned Python binary should be avoided as it may be Python2 or Python3 (depends on distribution). Implements: blueprint drop-py2-support Change-Id: Ifd7912a3debf3ee4ddd8f3e1a4c5870848d4d688
2019-07-21Change how build process treats unbuildable imagesMarcin Juszkiewicz1-1/+1
We have a list of images which are unbuildable for distro/buildtype/arch combos. This patch renames it from SKIPPED_IMAGES to UNBUILDABLE_IMAGES because 'skipped' images is something else in our code. Now all unbuildable images are marked as such and so are their children. At the end of build there is information provided about skipped images (due to --skip-parents or --skip-existing options) and then information about not buildable images. Both list should only contain entries related to requested build. Change-Id: If9b521339f564e483cba03d52e7c4eba271821a5
2019-03-26build: handle template syntax errors betterMarcin Juszkiewicz1-7/+13
Before: (venv-for-kolla) 15:35 (s) hrw@puchatek:kolla$ ./tools/build.py --template-only INFO:kolla.common.utils:Found the docker image folder at /home/hrw/devel/linaro/kolla/docker Traceback (most recent call last): [.. here goes 100 lines of Jinja2 traceback ..] jinja2.exceptions.TemplateSyntaxError: unexpected '}', expected ']' After: (venv-for-kolla) 15:44 (s) hrw@puchatek:kolla$ ./tools/build.py --template-only INFO:kolla.common.utils:Found the docker image folder at /home/hrw/devel/linaro/kolla/docker ERROR:kolla.common.utils:Syntax error in template: ironic-inspector/Dockerfile.j2 ERROR:kolla.common.utils:unexpected '}', expected ']' INFO:kolla.common.utils:Dockerfiles are generated in /tmp/kolla-2019-03-13_15-44-28_0ovk4xu_/docker Change-Id: Iaa899f0b81f6698aa86b15977c39af3269d31e9a
2018-12-27Update hacking versionconfi-surya1-1/+2
Use latest release 1.1.0 and compatible changes w.r.t pep8 Change-Id: Ia3d1166a2746a46591e4e5a04ea91e03ddc28d93
2017-03-29Allow the use of previously built parent imagesPaul Bourke1-1/+2
Consider the case where you have a set of tagged / released images to production, and you want to apply a small hotfix, security patch, etc. to one of them, say keystone-ssh. Rebuilding keystone-ssh will cause a rebuild of base, openstack-base, and keystone-base, potentially pulling in updated packages / new dependencies, unless you have access to the exact machine they were previously built on. Ideally you want to use the exact parent images that are in production. This patch adds a new build argument '--skip-parents', which will result in only the image(s) specified by the regex and their children to be built. Parents are expected to exist either on machine or in a registry. Change-Id: I79d5f0b422f48d2dc36ae85dfa21668cf8177837 Implements: blueprint image-build-depth
2016-10-14Consistently set sys path when using local toolsMartin André1-0/+3
Consistently set the search path to prefer PROJECT_ROOT as the source of packages to import if we are using local tools instead of pip installed kolla tools. TrivialFix Change-Id: Idd1d98bb82a8f1ddb32d1bdb23f346b461534b9f
2016-08-02Using sys.exit(main()) instead of main()Lu lei1-1/+1
As we known, Exceptions are raised by the sys.exit() function. When they are not handled, no stack traceback is printed in the Python interpreter. In this patch we have known main() got return values(eg:0, 1), but it can't specifies exit status when terminate the main thread by unusing sys.exit(). So when using sys.exit(main()) instead of main() may be more readable and reasonable. TrivialFix. Change-Id: I184289d28b92a7e345907247c045535a0c42c974
2016-06-13Change the search path to look locallyRyan Hallisey1-0/+8
In order to use tools/build.py, we need to search locally for imports. Closes-bug: #1592030 Change-Id: Idfa651c1268f93366de9f4e3fa80c33be42c71c3
2016-06-07Make build.py importable libJeffrey Zhang1-910/+2
First step to break it down is to make it importable lib rather than self-contained script. Change-Id: Ibbc924c2849cc79006046f62f31a1909ce959066 Partially-implements: bp build-refactor
2016-06-07Remove the deprecated kolla-build sectionJeffrey Zhang1-2/+1
TrivialFix Change-Id: I3ee10efc093ed394c3af483a229dcd9dc732ec9a
2016-06-02Use an image object, recorder object and status constantsJoshua Harlow1-117/+175
Instead of using raw dicts and passing data around via dictionaries (which makes it really hard to figure out what is in those dictionaries at any point) prefer to use objects. That way people can actually understand what the object is supposed to be, vs guessing and/or having to decipher its usage. The same goes for raw string constants, prefer using named constants instead. Closes-Bug: #1586475 Change-Id: Ide179dc6593c50696d47a2d3d4cd000f343855d4
2016-06-02Use a task model and dumb workersJoshua Harlow1-64/+136
Instead of having smart workers that need to know about the full logic of how they operate and what they will do which makes it hard to reuse parts of those workers switch the model so that workers are dumb and they just run tasks which themselves can request additional tasks to run when succesful. This makes it more consistent when tasks are to be retried and also makes it easier to know which task is running and when. Less specialized workers == good. Closes-Bug: #1586474 Change-Id: Ie7b7303b8bf2a80b3e26b6d4ffc27787f8c794d6
2016-06-01Merge "Don't always unpack image build results tuple, as it may be None"Jenkins1-5/+11
2016-06-01Merge "Pin the base distro release version"Jenkins1-5/+3
2016-05-31Don't always unpack image build results tuple, as it may be NoneMark Goddard1-5/+11
Change-Id: I21b1327da1de1bd3cc3018ba37715b0a19c1dbef Closes-Bug: #1587501
2016-05-31Pin the base distro release versionJeffrey Zhang1-5/+3
Pin the base distro release version in the master branch(Newton). Only one distro version is supported. The supported versions are: * CentOS: 7 * RedHat: 7 * OracleLinux: 7 * Debian: 8 * Ubuntu: 14.04 NOTE: Kolla will move to Ubuntu 16.04 in the Newton cycle DocImpact TrivialFix Change-Id: I32c0edbee4de14c8a0cc7c961d566e84d70f19a4
2016-05-27Fix kolla-build return codesWill Miller1-1/+9
Add a main wrapper function to return 1 if any images failed to build; otherwise, return 0. Rename the old main() to run_build(), which still returns image statuses, to avoid interfering with unit tests. Change-Id: I5817cc9cebb4f42dc13e0535223e5dc8931a3aab Co-Authored-By: Mark Goddard <mgoddard@cray.com> Closes-Bug: #1578404
2016-05-26Merge "Implement non-default signal handler for SIGINT"Jenkins1-5/+10
2016-05-10Merge "Catch all the exception in the push thread"Jenkins1-0/+3
2016-05-09Support a custom base imageJeffrey Zhang1-0/+1
Add a new option(base-image) to support custom base image In default, the base-image is equal to base. When using custom image, please use command line like: ./tool/build.py --base-image my-image --base centos --base-tag 0.1 DocImpact Closes-Bug: #1573544 Change-Id: If4b9c57a2b68f6f06d3e981f30b2173064e02193
2016-05-06Implement non-default signal handler for SIGINTJeff Peeler1-5/+10
Pressing Ctrl-C will stop a build, but does not clean up the /tmp directory of dockerfiles. Implement a signal handler such that clean up does occur. Python apparently ignores signals while calling join on a queue, so that's why the small modification was needed there. Change-Id: I43faf67726c154b6640d3596aea61ffc59871999 Closes-Bug: #1579242
2016-05-06Add images from profile as matched only if no regex is usedMichal Rostecki1-2/+1
Change-Id: I053e54c54883318dfc7daa64a58276487805de66 Closes-Bug: 1578955
2016-05-05Generate image dependencies on CLIJeff Peeler1-0/+40
Using the new --list-dependencies flag, one can query dependencies for a given filter. For example: ./build.py --list-dependencies heat swift {'base': [{'openstack-base': [{'heat-base': ['heat-engine', 'heat-api-cfn', 'heat-api']}, {'swift-base': ['swift-object', 'swift-proxy-server', 'swift-container', 'swift-rsyncd', 'swift-account']}]}]} Also added --list-images to list all available images. Change-Id: I1797e32e32705182a763f53329eeb5c4a361abec Implements: blueprint images-dependency-tree-cli
2016-05-02Adding a few #nosec tags to prepare for Bandit usageTravis McPeak1-1/+2
This commit adds a few #nosec tags for non-issues. I've examined the code and determined that these don't represent security issues so we should add a #nosec tag that tells Bandit not to find these issues in the future. Closes-Bug: #1577498 Change-Id: Ic37216c08442c700c64118c78cfb46e6cedd237c
2016-04-28Catch all the exception in the push threadJeffrey Zhang1-0/+3
Change-Id: I1ba48cd25e529bdd846d3ded1f5923f312e5d17d Closes-Bug: #1574931
2016-04-27Override default latest tag on ubuntuMichal (inc0) Jastrzebski1-0/+5
After 16.04 got released we can't build our images as we pull latest (16.04 now) and our sources lists are built for 14.04. We pin ubuntu to 14.04 here. Change-Id: Ib29a1c607c3a725cdaf2098edd1c4b80f06a5e38 Closes-Bug: #1575884
2016-04-05Log the SHA of the Git ref for source buildsPaul Bourke1-2/+3
When using named refs in source builds, e.g. "reference = master", one can't easily tell from the build output what SHA master actually pointed to at that point and time. Update the debug output to include this. Closes-Bug: #1566316 Change-Id: I95c1cc316ab7332fa30fa43e207a5a5512b76708
2016-03-09Add local directory or file for source buildSamYaple1-0/+10
This change allows using a local directory or tarball to build from sources by specifying the 'local' type in your kolla-build.conf: [keystone-base] type=local location=/home/martin/src/keystone [neutron-server-plugin-networking-cisco] type=local location=/tmp/networking-cisco.tar.gz Closes-Bug: #1509625 Co-Authored-By: Martin André <m.andre@redhat.com> Change-Id: I3f6f68b0f44efdac653d186d2625591ad7cb3b2a
2016-03-08Handle kolla-build exceptionSwapnil Kulkarni (coolsvap)1-2/+7
kolla-build currently does not provide information if docker service is not running. Added exception handling. Change-Id: I1f167804b8b4d2ad9220281954ff7ea7162a9817 Closes-Bug:#1553912
2016-02-28Add timeout to requests.get() in kolla/cmd/build.pyVikram Hosakote1-1/+8
When kolla-build is running, if there are network issues or if the source's location (like http://tarballs.openstack.org) fails to respond due to high number of concurrent requests, kolla-build just hangs/blocks indefinitely. This patch set resolves this issue by adding a timeout of 120 seconds for requests.get() in kolla/cmd/build.py, adds a unit test for it in kolla/tests/test_build.py and also the "timeout" argument in kolla/common/config.py. Change-Id: I7c8745a20b9bd1c3f5d6a55c72a794f16fd7e513 Closes-Bug: #1548614
2016-02-15fix the custom profile raise exception issueJeffrey Zhang1-1/+9
Closes-Bug: #1536272 Change-Id: I6c91e522923eade16ba23711e6014e6b4b8cd3e0
2016-02-12Fix image plugin functionality for oslo.configPaul Bourke1-1/+3
The new oslo.config style parsing for kolla-build.conf stopped plugins from working. This patch fixes it and adds a unit test. Co-Authored-By: Jeffrey Zhang <zhang.lei.fly@gmail.com> Closes-Bug: 1534556 Change-Id: I135797c733ae0cae040a009c364073769b19e4eb
2016-02-05Merge "Ubuntu custom apt repos"Jenkins1-0/+14
2016-02-04Move the install base type check to kolla-buildAllen Gao1-0/+15
The install type is converted in kolla-build, so it will never fail in dockerfile, move the check to kolla-build just above the install type converting. TrivialFix Co-Authored-By: Jeffrey Zhang <jeffrey.zhang@99cloud.net> Change-Id: I1500d3b47e909f94ea9f68c5245297733f63a70b
2016-02-03Ubuntu custom apt reposSamYaple1-0/+14
Change-Id: I3afb8a29c4ae2369f29e0cc94d75ff8a3c42594b Implements: blueprint custom-repos
2016-01-26Merge "Pick up proxy buildargs from environment"Jenkins1-1/+19
2016-01-20Import modules rather than objectsJeffrey Zhang1-10/+9
Base on the OpenStack Style Guidelines[0]: Do not import objects, only modules [0] http://docs.openstack.org/developer/hacking/index.html#imports TrivialFix Change-Id: Id1bc5e4b24fd561f1a9f6167af341b00a343a718
2016-01-18Pick up proxy buildargs from environmentPaul Bourke1-1/+19
We currently support any buildarg via --build-args. This patch picks up the default supported proxy buildargs if set in the user's environment, to allow for more transparent proxy support. The --build-args will take precendence. DocImpact Implements blueprint: better-proxy-support Change-Id: I084e5d1cc8f9993d65167878f9983ad58a68e86e
2016-01-18Expose the docker build_arg to build.pyJeffrey Zhang1-1/+2
With this implement, we can add variables at building stage. For example, add HTTP_PROXY and NO_PROXY when needed like below. build.py --build-args \ HTTP_PROXY:http://127.0.0.1:8080,NO_PROXY:127.0.0.1 More info about build_arg, pls check[0] [0] https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables-build-arg DocImpact Implements: bp add-buildargs Change-Id: I29ed7f536670fef59d697603dc562a69d90743c9
2016-01-16Use dict.items() for adding Python 3 compatibilityting.wang1-2/+2
Replacing dict.iteritems()/.itervalues() with six.iteritems(dict)/six.itervalues(dict) was preferred in the past, but there was a discussion suggesting to avoid six for this. ref: http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html Partially implements: blueprint python3 Change-Id: Ie931bc02f558f6db3677b44f645eb3bdf78ccae8
2016-01-08Merge "Fix the package source don't register issue"Jenkins1-7/+8
2016-01-07Fix the package source don't register issueJeffrey Zhang1-7/+8
Closes-Bug: #1531848 Change-Id: I3c6ceb9f1f367b8b19ab1b1d06c67817ab93cabb
2016-01-07Merge "Log information about successful push only if it really is"Jenkins1-1/+2
2016-01-07Fix broken local and global buildingSamYaple1-4/+13
Two things broke in the related patches. This patch aims to fix that. 1) tools/build.py was no longer usable as it imported kolla from the global pip install source. If that didnt exist (because kolla wasn't installed with pip) then it breaks on import. 2) When kolla was installed globally it looked up the path that matched the installation of docker-py instead of looking up a path that had the stuff we wanted (kolla images directory) Change-Id: I94f1856547ce54d506de72926b08c966a36ac608 Related-Id: I05f10d13e7ba1e2b985c2944aec71ce55630442b Related-Bug: #1530256 Closes-Bug: #1531569
2016-01-06Log information about successful push only if it really isMichal Rostecki1-1/+2
The try-except-finally block in push thread was catching only ConnectionError and logging information about successful push even if some other error occured during push. TrivialFix Change-Id: I94f5ed67237d291701469a65f76146881cd67a35
2016-01-06Merge "Fix the kolla to find the docker image folder in virtualenv"Jenkins1-25/+18
2016-01-06Gen the image dependency in the Graphviz dot formatJeffrey Zhang1-0/+20
By using ./tools/build.py --save-dependency a.dot to gen the Graphviz dot dependency file. Later, you can use the `dot` to draw the picture. dot -Tjpg -o a.jpg a.dot Implements: blueprint images-dependency-tree Depends-On: I8e07a1b69fab5f1c587470bfd2104aaba93f0050 Change-Id: If00f4f3fb9d0b10a07ab2abb7ffb1cd9d64902f2
2016-01-06Merge "Generate the kolla-build.conf file using oslo_config"Jenkins1-151/+7
2016-01-06Merge "Do not truncate the error messages"Jenkins1-1/+1
generated by cgit v1.2.3 (git 2.39.1) at 2026-11-05 19:33:51 +0000