summaryrefslogtreecommitdiff
path: root/docker/fluentd
AgeCommit message (Collapse)AuthorFilesLines
2020-09-03Upgrade Fluentd Monasca pluginDoug Szumski1-1/+1
Switch to YAJI JSON parser as suggested in [1]. [1] https://docs.fluentd.org/quickstart/faq#i-got-encoding-error-inside-plugin-how-to-fix-it Closes-Bug: #1889065 Change-Id: I2ecd3b029b956e000feaaa9e10a169521ba53d66 (cherry picked from commit 556d4353aa846765557cfdd3c6e087a25248c517)
2019-10-02Fluentd: There is available td-agent pkg for BusterViktor Michalek1-0/+6
Thanks a lot to Masahiro Nakagawa https://github.com/treasure-data/omnibus-td-agent/issues/209#issuecomment-530340475 Change-Id: I9c4993436844d45bc9df2815659fa7c420910b01
2019-09-25Debian: switch fluentd to td-agent on x86_64Radosław Piliszek1-7/+8
This patch uses the newly added td-agent repo on Debian. It also makes code more robust regarding different distro handling. We now have td-agent everywhere on x86_64. Change-Id: I4d2449d7d100c68084f937ef9981596fc1625375 Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-04Change fluentd to td-agent on CentOSMichal Nasiadka4-55/+54
Fluentd packages in CentOS Opstools SIG repos is very old, because of old-ish ruby (2.0) packaged with CentOS 7. On the other hand - TreasureData packages for fluentd are packaged with embedded ruby as dependencies - so this change moves to those. Thanks to that we are able to move from fluentd 0.12 to 1.4.2. Still we need ruby-devel for non-td-agent distros, because we download monasca output plugin as source and build it during installation. * Bump up td.repo to version 3 * Merge sudoers into one file * Change fluentd plugin install macro to reflect changes Depends-On: https://review.opendev.org/679209/ Depends-On: https://review.opendev.org/679885/ Change-Id: I40a81103317ac044d9311b2505593056cf313bd2
2019-08-22Merge "Add fluentd_binary and fluentd_version labels"Zuul1-0/+5
2019-08-19fluentd: there is no 'td-agent' for Debian/x86-64Marcin Juszkiewicz1-11/+7
As there is no 'td-agent' package image build fails on 'x86-64'. We have to wait for TD Agent 4. Change-Id: I03d38628eee0f0823a58cba3a70df22f29022956
2019-08-19Add fluentd_binary and fluentd_version labelsMichal Nasiadka1-0/+5
In order to orchestrate a smooth migration of fluentd to newer td-agent on CentOS x86_64, a label that points to fluentd binary and version are needed. Depends-On: https://review.opendev.org/677144/ Change-Id: Ie2257870d87c5b3a9b0572f8fd35085a3f95fedb
2019-08-19Unbreak CI: fluentd and TripleO dependencyRadosław Piliszek1-2/+0
fluentd - we are not deploying kubernetes, no need for plugins TripleO - CI currently broken (to reverse, FIXME in place) Change-Id: I203f70176af7f2b53398ddeb5c21e6234a45e268 Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-06-28Bump Monasca Fluentd output pluginDoug Szumski1-1/+1
This resolves an issue where the previous version of the plugin would fail to re-send buffered logs if a post to the Monasca Log API failed. With this new version of the plugin, logs are correctly buffered and resent. Change-Id: Ic1d93a8ea5f2571edfacf50809d51a8b77959a73
2019-05-29Use td-agent-gem to install fluentd monasca plugin on ubuntuMark Goddard1-1/+1
If monasca is enabled on ubuntu, we see the following error in fluentd: [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Unknown output plugin 'monasca'. Run 'gem search -rd fluent-plugin' to find plugins" This is because we use 'fluent-gem install' to install the gem, which works on CentOS but on Ubuntu it is necessary to use 'td-agent-gem install'. Change-Id: I515f9764ca93a41a871e520696193a9298856d08 Closes-Bug: #1830147
2019-05-28Merge "Install fluentd gem on debian/ubuntu for non x86_64"Zuul2-5/+23
2019-05-23Bump Monasca Fluentd plugin versionDoug Szumski1-1/+1
Bump version to include support for setting log message field name. Change-Id: Ib3953b49247bb36ac8ce25578c05c6bcb2a1905a Partial-Bug: #1830184
2019-03-28Install fluentd gem on debian/ubuntu for non x86_64Pedro Alvarez2-5/+23
The package used to install fluentd on debian/ubuntu is not available for architectures other than x86_64. For those cases install fluentd as a gem until the package 'td-agent' is available for more architectures. Change-Id: Ic1b25b37d06b429a112f1418a79d385d505bb1e9
2019-01-17Use base_package_typeAlex Schultz1-7/+7
This change updates the docker files to use base_package_type instead of doing specific distro checks for the rhel/deb generic cases. The base_distro is still available and is used when a specific distro needs a customization but if the differences are purely rpm vs deb, then the base_package_type can be used. Change-Id: I8d720bb185df65a0178061ccf20b1ab2265da2c5
2018-10-11Build and install Fluentd Monasca pluginDoug Szumski1-0/+16
This is a simple output plugin for Fluentd which can forward logs to the Monasca Log API. Partially-Implements: blueprint monasca-containers Change-Id: I599554052c3cb99ebcbcbc4d7ea1aeb57c573d2e
2018-01-05Fluentd plugins rpm installation on CentOSJuan Badia Payno1-24/+12
Currently, fluentd plugins are installed via gem install, this may causes some issues when gems are updated upstream. This patch uses the plugins packaged on CentOS opstools repository. The debian/ubuntu image has not been modified Change-Id: Ib32ceaae0d9fa56c6ec25a47ae6f44259d3b6e6d
2017-11-24Clean fluentd_plugins_install blockMartin Mágr1-16/+36
fluentd_plugins_install should contain only plugin installation operations, so that the override does not have to handle irrelevant operations Change-Id: I8f810d194a1e638683df1f445c187b8d98629730
2017-11-14Fix the Ubuntu Fluentd imagechenxing1-3/+1
For ubuntu, we actually use fluentd 0.14.22 and ruby 2.1.10 now. Thus, we should use the specific gem version for the plugins: - fluent-plugin-rewrite-tag-filter:2.0.0 According to the README.md of this plugin[1] It shows: ---------------------------------------------------------- | fluent-plugin-rewrite-tag-filter | Fluentd | Ruby | |----------------------------------|------------|--------| | >= 2.0.0 | >= v0.14.2 | >= 2.1 | | < 2.0.0 | >= v0.12.0 | >= 1.9 | ---------------------------------------------------------- - fluent-plugin-grok-parser:2.1.4 When using the fluent-plugin-rewrite-tag-filter 2.0.0 to fix this bug, another bug[1] which is the same as this one[2] was triggered. According to https://github.com/fluent/fluent-plugin-grok-parser, it shows: --------------------------------------------------- | fluent-plugin-grok-parser | fluentd | Ruby | |---------------------------|------------|--------| | >= 1.0.0 | >= v0.14.0 | >= 2.1 | | < 1.0.0 | >= v0.12.0 | >= 1.9 | --------------------------------------------------- We actually use fluentd 0.14.22 now, but the version of fluent-plugin-grok-parser is still pinned in 0.3.1. So we need to upgrade the fluent-plugin-grok-parser to 2.1.4. [1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter [2] http://paste.openstack.org/show/626145/ [3] https://github.com/fluent/fluent-plugin-grok-parser/issues/32 backport: pike Change-Id: I47985113fe732569d640a262ca832c6edc8c2bb6 Partial-Bug: #1730664
2017-11-06Fix CentOS 7 fluentd imageJuan Badia Payno1-1/+1
Use specific gem version for the plugins: - fluent-plugin-rewrite-tag-filter - fluent-plugin-elasticsearch Change-Id: Icc5b8972e7eca01767cdc9321ee82d1fcde187d8 Closes-Bug: 1730346
2017-10-16Revert "Fluentd: add fluent-plugin-concat:1.0.0 gem"Eduardo Gonzalez1-1/+1
This reverts commit 83baad304d359e17a5be3ad1e92f7cd31dc129e0. The plugin is not used in any deployment tool. Was introduced to make multiline but finally standard multiline was used. This change removes an unused plugin. Change-Id: Ie38a876d56846ebdec02fa05707e9d22a02788e6
2017-10-13Fluentd: enable forward logs over SSL with authenticationJuan Badia Payno1-2/+2
On the fluent Docker added the fluent-plugin-secure-forward, so fluentd will be able to forward logs over SSL with authentication Closes-Bug: #1723408 Change-Id: I231ebc1052dac93fb9420dcfb10595a3647c35ce
2017-08-29Merge "Fluentd: add fluent-plugin-concat:1.0.0 gem"Jenkins1-1/+1
2017-08-28fix image build error of fluentdjiangpch1-1/+1
the newest public_suffix gem required ruby>2.1.0, but the ruby on centos' release is 2.0.0, we should install public_suffix==2.0.5 manual to avoid the ruby version not matched error. Closes-Bug: #1713416 Change-Id: I725294da206a73b759027d3123d140bddafcfdfb
2017-08-17Fluentd: add fluent-plugin-concat:1.0.0 gemBertrand Lallau1-1/+1
Actual Fluentd config doesn't manage multiline log pattern like: 2017-08-16 21:41:03.097 7 WARNING oslo_config.cfg [req-34ea3f23-f780-4059-b911-2788ea4af7b2 - - - - -] Option "firewall_driver" from group "DEFAULT" is deprecated for removal ( nova-network is deprecated, as are any related configuration options. ). Its value may be silently ignored in the future. This log will generate 1 Fluentd records for each line (3 records), this is really a wrong behaviour, and made Kibana interface unusable. This patch add fluent-plugin-concat in order to concatenate multiline log separated in multiple events. Change-Id: I83e583605b49b7c421109d662de1291f38bd5c07 Closes-Bug: #1711399
2017-07-31Use community fluentd packages for CentOS/RHEL/OraclelinuxJuan Badia Payno3-8/+53
-Use the fluentd package on the CentOS OpsTools repository created by the CentOS OpsTools SIG. -The user changed to fluentd for CentOS/RHEL/Oraclelinux -Plugins are installed as gems on CentOS/RHEL/Oraclelinux Change-Id: I4802618373cfa53d67d1dfe65be7abe1950b3d04
2017-05-03Override image's meta info.Jawon Choo1-1/+1
centos based images have wrong label info, these changes fix own image's name and build-date. Change-Id: I1d13f8f386c8db12b5fbe5f8ecbbf9e3fbb4ba1c Closes-Bug: #1680341
2017-04-20Use LABEL instead of MAINTAINER (deprecated) in all Dockerfile.j2Chen1-1/+1
Use LABEL instruction instead of MAINTAINER (deprecated) instruc- tion as suggested by Docker's official dockerfile guide. docs.docker.com/engine/reference/builder/#maintainer-deprecated Closes-Bug: #1683652 Change-Id: Ie87a1ddf31aefcd0b623fd2837d78de420e76898
2017-04-04debian: enable all images enabled for UbuntuMarcin Juszkiewicz1-1/+1
Debian support is not maintained in Kolla so it got a bit behind Ubuntu one. This changeset enables Debian for all images. Jessie (even with backports) may be too old for some images though. Also unify distro check to ['debian', 'ubuntu'] to keep alphabetical order like it is done for RPM distributions. Partially-Implements: blueprint multiarch-and-arm64-containers Change-Id: I056233fbfa277e0e2360c07c3f80d9558c554357
2017-03-28Alphabetize packagesEduardo Gonzalez1-4/+4
Some images have packages sorted alphabetically and some not. Unify common style between all images. Change-Id: I906ed89c10b12886665618752f525ba71d83d991
2017-03-04Fix fluentd undefined method 'parse'zhubingbing1-1/+3
Now we use flunetd version is v12.0 we use fluent-plugin-grok-parser plugin shoud < 1.0.0. https://github.com/fluent/fluent-plugin-grok-parser Change-Id: I013e1e6d1070c4ba050be5dd75be7be4df652e4e Closes-Bug: #1669405
2017-02-15Fix the read mariadb log permission issue for fluentdJeffrey Zhang1-1/+1
mariadb.log mode is 660, which can not be read by fluentd. Add td-agent user into mysql group to allow fluent to read the log. Closes-Bug: #1664867 Change-Id: If624002909d5fcd0bb82a5bc9937945518ec84b9
2017-02-09Merge "Move fluentd td.repo to base for consistency"Jenkins2-13/+0
2017-02-09Merge "Add fluent-plugin-parser pluin"Jenkins1-1/+1
2017-02-09Add fluent-plugin-parser pluinzhubingbing1-1/+1
Change-Id: I4c22803f2d7fbea88ad2d2d4648cf498905d4013
2017-02-08Move fluentd td.repo to base for consistencyPaul Bourke2-13/+0
Additional repos for centos such as elasticsearch, ceph etc. are all set up in the base image, fluentd should follow this both for consistency and to make it easier for operators to remove/override. Change-Id: If2ee20232d72ccf6b1c4be06c2fb3d1fa219e9cc
2017-02-05Remove include_header and include_footer in all DockerfilesJeffrey Zhang1-1/+0
include_header and include_footer parameter is already removed, remove them in all Dockerfiles. Add missing footer block. Change-Id: I90da03eb9f95a3827361d5f5ede65fde7d6be2b3
2017-01-17use static uid/gid in imagesSam Yaple1-5/+4
This centralizes all user and group creation into a single source. This will fix any current and furture uid/gid mismatches (such as with nova-libvirt). In the process, we also unify users between the distros in a standard way. The users in the following containers change from thier defaults: Ubuntu: _chrony user is now chrony Ubuntu: memcache user is now memcached All: qemu user is used for ownership and socket permissions All uid and gid numbers are customizable via kolla-build.conf Co-Authored-By: Kris Lindgren <klindgren@godaddy.com> Change-Id: I120f26ab0683dc87d69727c3df8d4707e52a4543 Partially-Implements: blueprint static-uid-gid
2016-12-20Add fluent-plugin-rewrite-tag-filter in fluentdzhubingbing1-1/+1
we need use fluent-plugin-rewrite-tag-filter to classification openstack service Change-Id: I879b1bad744235de079a0498a76afcfe0bc664a6
2016-12-16Add fluent-plugin-grok-parser in fluentdzhubingbing1-1/+1
We use fluent-plugin-grok-parser to format openstack log in fluentd Change-Id: I4bc3de1a6f099dac2c6ea58f9b47545f0c2ce07e
2016-11-03Remove Fedora supportChristian Berendt1-1/+1
Closes-bug: #1616387 Change-Id: Id97f88b9baa3d48d33ce120962450a374282d044
2016-10-12Adds Fluentd docker image required for kolla-kubeSerguei Bezverkhi4-0/+79
This PS adds Fluentd docker image to Kolla. Kolla currently does not use Fluentd for log collection, but this image is required for Kubernetes centralized logging solution. Change-Id: I719661d75f5044239a1fd52080d322c77490ecae Partially_Implements: blueprint kubernetes-images
generated by cgit v1.2.3 (git 2.39.1) at 2026-11-05 23:14:00 +0000