aboutsummaryrefslogtreecommitdiff
path: root/kolla/cmd
AgeCommit message (Collapse)AuthorFilesLines
2021-01-20implement "allowed to fail" imagesMarcin Juszkiewicz1-2/+2
Sometimes we have images failing. Not always we care about them. So let allow them to fail and stop caring do they build or not. Change-Id: I01cf3635814255f682d6bd161cf1d985d92a9a31
2020-03-16do not use unversioned Python binaryMarcin Juszkiewicz1-1/+1
We said 'bye' to Python 2 already. So let us use Python 3 for all scripts. Unversioned Python binary should be avoided as it may be Python2 or Python3 (depends on distribution). Implements: blueprint drop-py2-support Change-Id: Ifd7912a3debf3ee4ddd8f3e1a4c5870848d4d688
2019-07-21Change how build process treats unbuildable imagesMarcin Juszkiewicz1-1/+1
We have a list of images which are unbuildable for distro/buildtype/arch combos. This patch renames it from SKIPPED_IMAGES to UNBUILDABLE_IMAGES because 'skipped' images is something else in our code. Now all unbuildable images are marked as such and so are their children. At the end of build there is information provided about skipped images (due to --skip-parents or --skip-existing options) and then information about not buildable images. Both list should only contain entries related to requested build. Change-Id: If9b521339f564e483cba03d52e7c4eba271821a5
2019-03-26build: handle template syntax errors betterMarcin Juszkiewicz1-7/+13
Before: (venv-for-kolla) 15:35 (s) hrw@puchatek:kolla$ ./tools/build.py --template-only INFO:kolla.common.utils:Found the docker image folder at /home/hrw/devel/linaro/kolla/docker Traceback (most recent call last): [.. here goes 100 lines of Jinja2 traceback ..] jinja2.exceptions.TemplateSyntaxError: unexpected '}', expected ']' After: (venv-for-kolla) 15:44 (s) hrw@puchatek:kolla$ ./tools/build.py --template-only INFO:kolla.common.utils:Found the docker image folder at /home/hrw/devel/linaro/kolla/docker ERROR:kolla.common.utils:Syntax error in template: ironic-inspector/Dockerfile.j2 ERROR:kolla.common.utils:unexpected '}', expected ']' INFO:kolla.common.utils:Dockerfiles are generated in /tmp/kolla-2019-03-13_15-44-28_0ovk4xu_/docker Change-Id: Iaa899f0b81f6698aa86b15977c39af3269d31e9a
2018-12-27Update hacking versionconfi-surya1-1/+2
Use latest release 1.1.0 and compatible changes w.r.t pep8 Change-Id: Ia3d1166a2746a46591e4e5a04ea91e03ddc28d93
2017-03-29Allow the use of previously built parent imagesPaul Bourke1-1/+2
Consider the case where you have a set of tagged / released images to production, and you want to apply a small hotfix, security patch, etc. to one of them, say keystone-ssh. Rebuilding keystone-ssh will cause a rebuild of base, openstack-base, and keystone-base, potentially pulling in updated packages / new dependencies, unless you have access to the exact machine they were previously built on. Ideally you want to use the exact parent images that are in production. This patch adds a new build argument '--skip-parents', which will result in only the image(s) specified by the regex and their children to be built. Parents are expected to exist either on machine or in a registry. Change-Id: I79d5f0b422f48d2dc36ae85dfa21668cf8177837 Implements: blueprint image-build-depth
2016-12-29Clean up kolla-ansible related files from KollaSayantani Goswami1-96/+0
- Remove globals.yml and passwords.yml files. - The gate was still using these files from the kolla directory. Modified the gate to prevent using these files from the kolla directory. - Modified the deploy_aio.sh file to populate passwords in passwords.yml using kolla-ansible. Change-Id: I1ed7849d54cab6d5a9217dced73327ea13f06636 Closes-Bug: #1653035
2016-11-30Replaces uuid.uuid4 with uuidutils.generate_uuid()caoyuan1-2/+2
Openstack common has a wrapper for generating uuids. We should only use that function when generating uuids for consistency. Fixed bug #1082248 Change-Id: I9db4da4fb774109720abb0852fad30cc8e17aafb
2016-10-14Consistently set sys path when using local toolsMartin André2-0/+12
Consistently set the search path to prefer PROJECT_ROOT as the source of packages to import if we are using local tools instead of pip installed kolla tools. TrivialFix Change-Id: Idd1d98bb82a8f1ddb32d1bdb23f346b461534b9f
2016-09-06Add gnocchi ansible rolezhubingbing1-1/+3
Partially-Implements: blueprint ansible-gnocchi Change-Id: I8dd0460bd21ac0a233fab0142ec7b6079459bdc2
2016-08-31adds bifrost ssh key generationSean Mooney1-1/+2
- This change extend the genpwd.py command to generate an ssh key pair bifrost. - This change bifrost config and bootstrap task to install the generated keys. - This change updates the bifrost guide to discribe how to provide your own key. Change-Id: I05243f58843d9195cace253dff5628fae89c78e8 Implements: blueprint bifrost-support
2016-08-28Merge "Fix bandit gate jobs"Jenkins1-2/+2
2016-08-28Fix bandit gate jobsPaul Bourke1-2/+2
* Inspected each error and fixed / added nosec where appropriate. * build-swift-ring.py which was throwing sec errors is no longer used so removed it. * Removed the dev/ directory from being checked. Closes-Bug: #1617713 Change-Id: I25664cabca4137e5c9f499c1af3f5ce78b86fb56
2016-08-25Add full support for fernetShaun Smekel1-1/+1
This addresses the ansible aspects of fernet key bootstrapping as well as distributed key rotation. - Bootstrapping is handled in the same way as keystone bootstrap. - A new keystone-fernet and keystone-ssh container is created to allow the nodes to communicate with each other (taken from nova-ssh). - The keystone-fernet is a keystone container with crontab installed. This will handle key rotations through keystone-manage and trigger an rsync to push new tokens to other nodes. - Key rotation is setup to be balanced across the keystone nodes using a round-robbin style. This ensures that any node failures will not stop the keys from rotating. This is configured by a desired token expiration time which then determines the cron scheduling for each node as well as the number of fernet tokens in rotation. - Ability for recovered node to resync with the cluster. When a node starts it will run sanity checks to ensure that its fernet tokens are not stale. If they are it will rsync with other nodes to ensure its tokens are up to date. The Docker component is implemented in: https://review.openstack.org/#/c/349366 Change-Id: I15052c25a1d1149d364236f10ced2e2346119738 Implements: blueprint keystone-fernet-token
2016-08-10add baremetal role to install kolla deps.Sean Mooney1-1/+1
Change-Id: Ie70db1b18a73528b1194e4fbcf53d09fed20f4a0 Implements: blueprint kolla-host
2016-08-02Using sys.exit(main()) instead of main()Lu lei1-1/+1
As we known, Exceptions are raised by the sys.exit() function. When they are not handled, no stack traceback is printed in the Python interpreter. In this patch we have known main() got return values(eg:0, 1), but it can't specifies exit status when terminate the main thread by unusing sys.exit(). So when using sys.exit(main()) instead of main() may be more readable and reasonable. TrivialFix. Change-Id: I184289d28b92a7e345907247c045535a0c42c974
2016-06-13Change the search path to look locallyRyan Hallisey1-0/+8
In order to use tools/build.py, we need to search locally for imports. Closes-bug: #1592030 Change-Id: Idfa651c1268f93366de9f4e3fa80c33be42c71c3
2016-06-07Make build.py importable libJeffrey Zhang1-910/+2
First step to break it down is to make it importable lib rather than self-contained script. Change-Id: Ibbc924c2849cc79006046f62f31a1909ce959066 Partially-implements: bp build-refactor
2016-06-07Remove the deprecated kolla-build sectionJeffrey Zhang1-2/+1
TrivialFix Change-Id: I3ee10efc093ed394c3af483a229dcd9dc732ec9a
2016-06-02Use an image object, recorder object and status constantsJoshua Harlow1-117/+175
Instead of using raw dicts and passing data around via dictionaries (which makes it really hard to figure out what is in those dictionaries at any point) prefer to use objects. That way people can actually understand what the object is supposed to be, vs guessing and/or having to decipher its usage. The same goes for raw string constants, prefer using named constants instead. Closes-Bug: #1586475 Change-Id: Ide179dc6593c50696d47a2d3d4cd000f343855d4
2016-06-02Use a task model and dumb workersJoshua Harlow1-64/+136
Instead of having smart workers that need to know about the full logic of how they operate and what they will do which makes it hard to reuse parts of those workers switch the model so that workers are dumb and they just run tasks which themselves can request additional tasks to run when succesful. This makes it more consistent when tasks are to be retried and also makes it easier to know which task is running and when. Less specialized workers == good. Closes-Bug: #1586474 Change-Id: Ie7b7303b8bf2a80b3e26b6d4ffc27787f8c794d6
2016-06-01Merge "Don't always unpack image build results tuple, as it may be None"Jenkins1-5/+11
2016-06-01Merge "Pin the base distro release version"Jenkins1-5/+3
2016-05-31Don't always unpack image build results tuple, as it may be NoneMark Goddard1-5/+11
Change-Id: I21b1327da1de1bd3cc3018ba37715b0a19c1dbef Closes-Bug: #1587501
2016-05-31Pin the base distro release versionJeffrey Zhang1-5/+3
Pin the base distro release version in the master branch(Newton). Only one distro version is supported. The supported versions are: * CentOS: 7 * RedHat: 7 * OracleLinux: 7 * Debian: 8 * Ubuntu: 14.04 NOTE: Kolla will move to Ubuntu 16.04 in the Newton cycle DocImpact TrivialFix Change-Id: I32c0edbee4de14c8a0cc7c961d566e84d70f19a4
2016-05-27Fix kolla-build return codesWill Miller1-1/+9
Add a main wrapper function to return 1 if any images failed to build; otherwise, return 0. Rename the old main() to run_build(), which still returns image statuses, to avoid interfering with unit tests. Change-Id: I5817cc9cebb4f42dc13e0535223e5dc8931a3aab Co-Authored-By: Mark Goddard <mgoddard@cray.com> Closes-Bug: #1578404
2016-05-26Merge "Implement non-default signal handler for SIGINT"Jenkins1-5/+10
2016-05-21Make passwords.yml file generation configurableSwapnil Kulkarni (coolsvap)1-2/+12
partially implements blueprint multiple-clouds Change-Id: I676c4245e6f058ffbed345970ee78d1750dd0f2f
2016-05-10Merge "Catch all the exception in the push thread"Jenkins1-0/+3
2016-05-09Support a custom base imageJeffrey Zhang1-0/+1
Add a new option(base-image) to support custom base image In default, the base-image is equal to base. When using custom image, please use command line like: ./tool/build.py --base-image my-image --base centos --base-tag 0.1 DocImpact Closes-Bug: #1573544 Change-Id: If4b9c57a2b68f6f06d3e981f30b2173064e02193
2016-05-06Implement non-default signal handler for SIGINTJeff Peeler1-5/+10
Pressing Ctrl-C will stop a build, but does not clean up the /tmp directory of dockerfiles. Implement a signal handler such that clean up does occur. Python apparently ignores signals while calling join on a queue, so that's why the small modification was needed there. Change-Id: I43faf67726c154b6640d3596aea61ffc59871999 Closes-Bug: #1579242
2016-05-06Add images from profile as matched only if no regex is usedMichal Rostecki1-2/+1
Change-Id: I053e54c54883318dfc7daa64a58276487805de66 Closes-Bug: 1578955
2016-05-05Generate image dependencies on CLIJeff Peeler1-0/+40
Using the new --list-dependencies flag, one can query dependencies for a given filter. For example: ./build.py --list-dependencies heat swift {'base': [{'openstack-base': [{'heat-base': ['heat-engine', 'heat-api-cfn', 'heat-api']}, {'swift-base': ['swift-object', 'swift-proxy-server', 'swift-container', 'swift-rsyncd', 'swift-account']}]}]} Also added --list-images to list all available images. Change-Id: I1797e32e32705182a763f53329eeb5c4a361abec Implements: blueprint images-dependency-tree-cli
2016-05-02Adding a few #nosec tags to prepare for Bandit usageTravis McPeak1-1/+2
This commit adds a few #nosec tags for non-issues. I've examined the code and determined that these don't represent security issues so we should add a #nosec tag that tells Bandit not to find these issues in the future. Closes-Bug: #1577498 Change-Id: Ic37216c08442c700c64118c78cfb46e6cedd237c
2016-04-28Catch all the exception in the push threadJeffrey Zhang1-0/+3
Change-Id: I1ba48cd25e529bdd846d3ded1f5923f312e5d17d Closes-Bug: #1574931
2016-04-27Override default latest tag on ubuntuMichal (inc0) Jastrzebski1-0/+5
After 16.04 got released we can't build our images as we pull latest (16.04 now) and our sources lists are built for 14.04. We pin ubuntu to 14.04 here. Change-Id: Ib29a1c607c3a725cdaf2098edd1c4b80f06a5e38 Closes-Bug: #1575884
2016-04-05Log the SHA of the Git ref for source buildsPaul Bourke1-2/+3
When using named refs in source builds, e.g. "reference = master", one can't easily tell from the build output what SHA master actually pointed to at that point and time. Update the debug output to include this. Closes-Bug: #1566316 Change-Id: I95c1cc316ab7332fa30fa43e207a5a5512b76708
2016-04-03Implement nova-ssh containerJeffrey Zhang1-0/+23
Add a nova-ssh container to handle the `nova migrate` and `nova resize` case, in which the nova will use ssh to copy files between machines. Change-Id: Ie6675943f3aeabfbba8589d308d55b9c89d732db Closes-Bug: #1562141
2016-03-21Add generate_passwords.py to generate passwordsSamYaple1-0/+51
As with all tools, this is a first pass at the generation. Perhaps we even want to move this into kolla/kolla/cmd and be generated with tox itself in the future. This tool, when run, will only populate empty fields that have no values meaning that it is safe to run repeatedly on the same file. Of note, there is no way to preserve comments in the file after it has been processed by the yaml parser in python. Comments and sections will remain in the passwords.yml template for additional documentation if the user wishes to populate the file themselves. Use SystemRandom and clean up the docs a bit to not use pronouns. Co-Authored-By: Steven Dake <stdake@cisco.com> Closes-Bug: #1559266 Change-Id: I2932d592df8871f1b7811059206d0b4d0553a687
2016-03-09Add local directory or file for source buildSamYaple1-0/+10
This change allows using a local directory or tarball to build from sources by specifying the 'local' type in your kolla-build.conf: [keystone-base] type=local location=/home/martin/src/keystone [neutron-server-plugin-networking-cisco] type=local location=/tmp/networking-cisco.tar.gz Closes-Bug: #1509625 Co-Authored-By: Martin André <m.andre@redhat.com> Change-Id: I3f6f68b0f44efdac653d186d2625591ad7cb3b2a
2016-03-08Handle kolla-build exceptionSwapnil Kulkarni (coolsvap)1-2/+7
kolla-build currently does not provide information if docker service is not running. Added exception handling. Change-Id: I1f167804b8b4d2ad9220281954ff7ea7162a9817 Closes-Bug:#1553912
2016-02-28Add timeout to requests.get() in kolla/cmd/build.pyVikram Hosakote1-1/+8
When kolla-build is running, if there are network issues or if the source's location (like http://tarballs.openstack.org) fails to respond due to high number of concurrent requests, kolla-build just hangs/blocks indefinitely. This patch set resolves this issue by adding a timeout of 120 seconds for requests.get() in kolla/cmd/build.py, adds a unit test for it in kolla/tests/test_build.py and also the "timeout" argument in kolla/common/config.py. Change-Id: I7c8745a20b9bd1c3f5d6a55c72a794f16fd7e513 Closes-Bug: #1548614
2016-02-15fix the custom profile raise exception issueJeffrey Zhang1-1/+9
Closes-Bug: #1536272 Change-Id: I6c91e522923eade16ba23711e6014e6b4b8cd3e0
2016-02-12Fix image plugin functionality for oslo.configPaul Bourke1-1/+3
The new oslo.config style parsing for kolla-build.conf stopped plugins from working. This patch fixes it and adds a unit test. Co-Authored-By: Jeffrey Zhang <zhang.lei.fly@gmail.com> Closes-Bug: 1534556 Change-Id: I135797c733ae0cae040a009c364073769b19e4eb
2016-02-05Merge "Ubuntu custom apt repos"Jenkins1-0/+14
2016-02-04Move the install base type check to kolla-buildAllen Gao1-0/+15
The install type is converted in kolla-build, so it will never fail in dockerfile, move the check to kolla-build just above the install type converting. TrivialFix Co-Authored-By: Jeffrey Zhang <jeffrey.zhang@99cloud.net> Change-Id: I1500d3b47e909f94ea9f68c5245297733f63a70b
2016-02-03Ubuntu custom apt reposSamYaple1-0/+14
Change-Id: I3afb8a29c4ae2369f29e0cc94d75ff8a3c42594b Implements: blueprint custom-repos
2016-01-26Merge "Pick up proxy buildargs from environment"Jenkins1-1/+19
2016-01-20Import modules rather than objectsJeffrey Zhang1-10/+9
Base on the OpenStack Style Guidelines[0]: Do not import objects, only modules [0] http://docs.openstack.org/developer/hacking/index.html#imports TrivialFix Change-Id: Id1bc5e4b24fd561f1a9f6167af341b00a343a718
2016-01-18Pick up proxy buildargs from environmentPaul Bourke1-1/+19
We currently support any buildarg via --build-args. This patch picks up the default supported proxy buildargs if set in the user's environment, to allow for more transparent proxy support. The --build-args will take precendence. DocImpact Implements blueprint: better-proxy-support Change-Id: I084e5d1cc8f9993d65167878f9983ad58a68e86e
generated by cgit v1.2.3 (git 2.39.1) at 2026-11-05 19:45:13 +0000