Convert Neutron to thin containers

*** Requires Docker 1.10 which is released ***

Documentation will be in the next patch. You must set the following
in your docker.service daemon control file for propogation to work:

    [Service]
    MountFlags=shared

======================================================================

Thanks to mount propagation in Docker 1.10 we can use thin containers
finally! This is extremely useful to operators since now they can
access the network namespaces from the hosts (outside the neutron
container). But additionally it allows us to implement the VPN agent
and other services easier.

Neutron containers and the neutron role are brought into the standards
of the new Kolla. Completely with drop-root and ansible formating
updates.

The ip_wrapper.py script was (thankfully) not needed so it has been
removed from the repo.

Partially-Implements: blueprint upgrade-neutron
Change-Id: Iaf5555283240457e1912459f397a6393d886fba1

20 files changed, 75 insertions, 187 deletions

diff --git a/docker/neutron/neutron-agents/Dockerfile.j2 b/docker/neutron/neutron-agents/Dockerfile.j2
deleted file mode 100644
index ce4b7202d..000000000
--- a/docker/neutron/neutron-agents/Dockerfile.j2
+++ /dev/null
@@ -1,52 +0,0 @@
-FROM {{ namespace }}/{{ image_prefix }}neutron-base:{{ tag }}
-MAINTAINER {{ maintainer }}
-
-{% if base_distro in ['centos', 'fedora', 'oraclelinux', 'rhel'] %}
-
-RUN yum install -y \
-        openvswitch \
-        supervisor \
-    && yum clean all
-
-COPY supervisord.conf /etc/
-
-{% elif base_distro in ['ubuntu', 'debian'] %}
-
-RUN apt-get install -y --no-install-recommends supervisor \
-    && apt-get clean
-
-COPY supervisord.conf /etc/supervisor/
-
-    {% if install_type == 'binary' %}
-
-RUN apt-get install -y --no-install-recommends \
-        neutron-metadata-agent \
-        neutron-dhcp-agent \
-        neutron-l3-agent \
-    && apt-get clean
-
-    {% endif %}
-{% endif %}
-
-# TODO: SamYaple FWaaS is part of the l3-agent, not a separate agent that is
-# why this file is needed. To support FWaaS we cannot have a separate container
-# I need to figure out the best way to make this work together
-#
-# This file _does_ _not_ need to exist, you must remove referencing it from the
-# exec line in the start script. Also all these config options can exist in the
-# main neutron.conf if we wish
-COPY fwaas_driver.ini /etc/neutron/
-
-COPY neutron-dhcp-agent/ /var/lib/kolla/neutron-dhcp-agent
-COPY neutron-l3-agent/ /var/lib/kolla/neutron-l3-agent
-COPY neutron-metadata-agent/ /var/lib/kolla/neutron-metadata-agent
-COPY config-sudoers.sh /var/lib/kolla/
-
-COPY start.sh /
-
-# We will remove this line in Docker 1.10 when namespace propagation works
-VOLUME /run/netns
-
-CMD ["/start.sh"]
-
-{{ include_footer }}
diff --git a/docker/neutron/neutron-agents/config-sudoers.sh b/docker/neutron/neutron-agents/config-sudoers.sh
deleted file mode 100644
index 82cf0d624..000000000
--- a/docker/neutron/neutron-agents/config-sudoers.sh
+++ /dev/null
@@ -1,12 +0,0 @@
-#!/bin/bash
-
-# Neutron uses rootwrap which requires a tty for sudo.
-# Since the container is running in daemon mode, a tty
-# is not present and requiretty must be commented out.
-if [ ! -f /sudo-modified ]; then
-    chmod 0640 /etc/sudoers
-    sed -i '/Defaults    requiretty/s/^/#/' /etc/sudoers
-    chmod 0440 /etc/sudoers
-fi
-
-touch /sudo-modified
diff --git a/docker/neutron/neutron-agents/fwaas_driver.ini b/docker/neutron/neutron-agents/fwaas_driver.ini
deleted file mode 120000
index a9b1ec519..000000000
--- a/docker/neutron/neutron-agents/fwaas_driver.ini
+++ /dev/null
@@ -1 +0,0 @@
-neutron-l3-agent/fwaas_driver.ini
\ No newline at end of file
diff --git a/docker/neutron/neutron-agents/neutron-dhcp-agent/start.sh b/docker/neutron/neutron-agents/neutron-dhcp-agent/start.sh
deleted file mode 100755
index 47c2ee160..000000000
--- a/docker/neutron/neutron-agents/neutron-dhcp-agent/start.sh
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/bin/bash
-set -o errexit
-
-# Loading common functions.
-source /var/lib/kolla/config-sudoers.sh
-
-# Will be removed when neutron-agents is a thin container
-CMD="neutron-dhcp-agent"
-ARGS="--config-file /etc/neutron/neutron.conf --config-file /etc/neutron/dhcp_agent.ini"
-
-exec $CMD $ARGS
diff --git a/docker/neutron/neutron-agents/neutron-l3-agent/fwaas_driver.ini b/docker/neutron/neutron-agents/neutron-l3-agent/fwaas_driver.ini
deleted file mode 100644
index 41f761abf..000000000
--- a/docker/neutron/neutron-agents/neutron-l3-agent/fwaas_driver.ini
+++ /dev/null
@@ -1,3 +0,0 @@
-[fwaas]
-#driver = neutron.services.firewall.drivers.linux.iptables_fwaas.IptablesFwaasDriver
-#enabled = True
diff --git a/docker/neutron/neutron-agents/neutron-l3-agent/start.sh b/docker/neutron/neutron-agents/neutron-l3-agent/start.sh
deleted file mode 100755
index da1b1cc57..000000000
--- a/docker/neutron/neutron-agents/neutron-l3-agent/start.sh
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/bin/bash
-set -o errexit
-
-# Loading common functions.
-source /var/lib/kolla/config-sudoers.sh
-
-# Will be removed when this container is broken out into thin containers
-CMD="neutron-l3-agent"
-ARGS="--config-file /etc/neutron/neutron.conf --config-file /etc/neutron/l3_agent.ini --config-file /etc/neutron/fwaas_driver.ini --config-file /etc/neutron/plugins/ml2/ml2_conf.ini"
-
-exec $CMD $ARGS
diff --git a/docker/neutron/neutron-agents/neutron-metadata-agent/start.sh b/docker/neutron/neutron-agents/neutron-metadata-agent/start.sh
deleted file mode 100755
index 9464a5196..000000000
--- a/docker/neutron/neutron-agents/neutron-metadata-agent/start.sh
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/bin/bash
-set -o errexit
-
-# Loading common functions.
-source /var/lib/kolla/config-sudoers.sh
-
-# Will be removed when this container is broken out in thin containers
-CMD="neutron-metadata-agent"
-ARGS="--config-file /etc/neutron/neutron.conf --config-file /etc/neutron/metadata_agent.ini"
-
-exec $CMD $ARGS
diff --git a/docker/neutron/neutron-agents/start.sh b/docker/neutron/neutron-agents/start.sh
deleted file mode 100755
index fb1732277..000000000
--- a/docker/neutron/neutron-agents/start.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-set -o errexit
-
-# We must remove all of the stale namespaces if they exist
-rm -f /run/netns/*
-
-# Processing /var/lib/kolla/config_files/config.json
-python /usr/local/bin/kolla_set_configs
-
-exec /usr/bin/supervisord
diff --git a/docker/neutron/neutron-agents/supervisord.conf b/docker/neutron/neutron-agents/supervisord.conf
deleted file mode 100644
index b7a808b3a..000000000
--- a/docker/neutron/neutron-agents/supervisord.conf
+++ /dev/null
@@ -1,40 +0,0 @@
-[unix_http_server]
-file = /var/run/supervisor.sock
-
-[inet_http_server]
-port = 127.0.0.1:9001
-
-[supervisord]
-nodaemon=true
-logfile = /var/log/supervisor/supervisord.log
-logfile_maxbytes = 200KB
-logfile_backups = 1
-pidfile = /var/run/supervisord.pid
-childlogdir = /var/log/supervisor
-
-[rpcinterface:supervisor]
-supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
-
-[supervisorctl]
-serverurl = unix:///var/run/supervisor.sock
-
-[program:neutron-dhcp-agent]
-command=/var/lib/kolla/neutron-dhcp-agent/start.sh
-priority=40
-startsec=10
-stderr_events_enabled=true
-stdout_events_enabled=true
-
-[program:neutron-metadata-agent]
-command=/var/lib/kolla/neutron-metadata-agent/start.sh
-priority=40
-startsec=10
-stderr_events_enabled=true
-stdout_events_enabled=true
-
-[program:neutron-l3-agent]
-command=/var/lib/kolla/neutron-l3-agent/start.sh
-priority=40
-startsec=10
-stderr_events_enabled=true
-stdout_events_enabled=true
diff --git a/docker/neutron/neutron-base/Dockerfile.j2 b/docker/neutron/neutron-base/Dockerfile.j2
index 82b70aaa4..7eed86b92 100644
--- a/docker/neutron/neutron-base/Dockerfile.j2
+++ b/docker/neutron/neutron-base/Dockerfile.j2
@@ -36,11 +36,12 @@ RUN yum -y install \
     {% elif base_distro in ['ubuntu', 'debian'] %}
 
 RUN apt-get install -y --no-install-recommends \
-        iptables \
+        arping \
         dnsmasq \
-        uuid-runtime \
         ipset \
+        iptables \
         openvswitch-switch \
+        uuid-runtime \
     && apt-get clean
 
     {% endif %}
@@ -49,15 +50,17 @@ ADD neutron-base-archive /neutron-base-source
 RUN ln -s neutron-base-source/* neutron \
     && useradd --user-group neutron \
     && /var/lib/kolla/venv/bin/pip --no-cache-dir install --upgrade -c requirements/upper-constraints.txt /neutron \
-    && mkdir -p /etc/neutron /usr/share/neutron /var/log/neutron /home/neutron \
+    && mkdir -p /etc/neutron /usr/share/neutron /var/lib/neutron /var/log/neutron /home/neutron \
     && cp -r /neutron/etc/* /etc/neutron/ \
     && cp -r /neutron/etc/neutron/* /etc/neutron/ \
     && cp /neutron/etc/api-paste.ini /usr/share/neutron \
     && mv /etc/neutron/neutron/ /etc/neutron/plugins/ \
-    && chown -R neutron: /etc/neutron /usr/share/neutron /var/log/neutron /home/neutron \
+    && chown -R neutron: /etc/neutron /usr/share/neutron /var/lib/neutron /var/log/neutron /home/neutron \
     && sed -i 's|^exec_dirs.*|exec_dirs=/var/lib/kolla/venv/bin,/sbin,/usr/sbin,/bin,/usr/bin,/usr/local/bin,/usr/local/sbin|g' /etc/neutron/rootwrap.conf
 
 {% endif %}
 
-COPY extend_start.sh /usr/local/bin/kolla_extend_start
-RUN chmod 755 /usr/local/bin/kolla_extend_start
+COPY neutron_sudoers /etc/sudoers.d/neutron_sudoers
+RUN usermod -a -G kolla neutron \
+    && chmod 750 /etc/sudoers.d \
+    && chmod 440 /etc/sudoers.d/neutron_sudoers
diff --git a/docker/neutron/neutron-base/extend_start.sh b/docker/neutron/neutron-base/extend_start.sh
deleted file mode 100644
index 07212efce..000000000
--- a/docker/neutron/neutron-base/extend_start.sh
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/bin/bash
-
-# Neutron uses rootwrap which requires a tty for sudo.
-# Since the container is running in daemon mode, a tty
-# is not present and requiretty must be commented out.
-if [ ! -f /sudo-modified ]; then
-    chmod 0640 /etc/sudoers
-    sed -i '/Defaults    requiretty/s/^/#/' /etc/sudoers
-    chmod 0440 /etc/sudoers
-    touch /sudo-modified
-fi
diff --git a/docker/neutron/neutron-base/neutron_sudoers b/docker/neutron/neutron-base/neutron_sudoers
new file mode 100644
index 000000000..ad4489200
--- /dev/null
+++ b/docker/neutron/neutron-base/neutron_sudoers
@@ -0,0 +1 @@
+neutron ALL = (root) NOPASSWD: /var/lib/kolla/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf *
diff --git a/docker/neutron/neutron-dhcp-agent/Dockerfile.j2 b/docker/neutron/neutron-dhcp-agent/Dockerfile.j2
new file mode 100644
index 000000000..ea5ede23c
--- /dev/null
+++ b/docker/neutron/neutron-dhcp-agent/Dockerfile.j2
@@ -0,0 +1,16 @@
+FROM {{ namespace }}/{{ image_prefix }}neutron-base:{{ tag }}
+MAINTAINER {{ maintainer }}
+
+{% if base_distro in ['ubuntu', 'debian'] %}
+    {% if install_type == 'binary' %}
+
+RUN apt-get install -y --no-install-recommends \
+        neutron-dhcp-agent \
+    && apt-get clean
+
+    {% endif %}
+{% endif %}
+
+{{ include_footer }}
+
+USER neutron
diff --git a/docker/neutron/neutron-l3-agent/Dockerfile.j2 b/docker/neutron/neutron-l3-agent/Dockerfile.j2
new file mode 100644
index 000000000..be3087d53
--- /dev/null
+++ b/docker/neutron/neutron-l3-agent/Dockerfile.j2
@@ -0,0 +1,16 @@
+FROM {{ namespace }}/{{ image_prefix }}neutron-base:{{ tag }}
+MAINTAINER {{ maintainer }}
+
+{% if base_distro in ['ubuntu', 'debian'] %}
+    {% if install_type == 'binary' %}
+
+RUN apt-get install -y --no-install-recommends \
+        neutron-l3-agent \
+    && apt-get clean
+
+    {% endif %}
+{% endif %}
+
+{{ include_footer }}
+
+USER neutron
diff --git a/docker/neutron/neutron-metadata-agent/Dockerfile.j2 b/docker/neutron/neutron-metadata-agent/Dockerfile.j2
new file mode 100644
index 000000000..f8798e93c
--- /dev/null
+++ b/docker/neutron/neutron-metadata-agent/Dockerfile.j2
@@ -0,0 +1,23 @@
+FROM {{ namespace }}/{{ image_prefix }}neutron-base:{{ tag }}
+MAINTAINER {{ maintainer }}
+
+{% if base_distro in ['ubuntu', 'debian'] %}
+    {% if install_type == 'binary' %}
+
+RUN apt-get install -y --no-install-recommends \
+        neutron-metadata-agent \
+    && apt-get clean
+
+    {% endif %}
+{% endif %}
+
+COPY neutron_sudoers /etc/sudoers.d/neutron_sudoers
+COPY extend_start.sh /usr/local/bin/kolla_extend_start
+RUN chmod 755 /usr/local/bin/kolla_extend_start \
+    && chmod 750 /etc/sudoers.d \
+    && chmod 440 /etc/sudoers.d/neutron_sudoers \
+    && usermod -a -G kolla neutron
+
+{{ include_footer }}
+
+USER neutron
diff --git a/docker/neutron/neutron-metadata-agent/extend_start.sh b/docker/neutron/neutron-metadata-agent/extend_start.sh
new file mode 100644
index 000000000..c46f60f55
--- /dev/null
+++ b/docker/neutron/neutron-metadata-agent/extend_start.sh
@@ -0,0 +1,6 @@
+#!/bin/bash
+
+# Only update permissions if permissions need to be updated
+if [[ $(stat -c %U:%G /var/lib/neutron/kolla) != "neutron:neutron" ]]; then
+    sudo chown neutron: /var/lib/neutron/kolla
+fi
diff --git a/docker/neutron/neutron-metadata-agent/neutron_sudoers b/docker/neutron/neutron-metadata-agent/neutron_sudoers
new file mode 100644
index 000000000..17b8b3dee
--- /dev/null
+++ b/docker/neutron/neutron-metadata-agent/neutron_sudoers
@@ -0,0 +1 @@
+%kolla ALL=(root) NOPASSWD: /bin/chown neutron\: /var/lib/neutron/kolla, /usr/bin/chown neutron\: /var/lib/neutron/kolla
diff --git a/docker/neutron/neutron-openvswitch-agent/Dockerfile.j2 b/docker/neutron/neutron-openvswitch-agent/Dockerfile.j2
index de075da25..762b1012b 100644
--- a/docker/neutron/neutron-openvswitch-agent/Dockerfile.j2
+++ b/docker/neutron/neutron-openvswitch-agent/Dockerfile.j2
@@ -19,14 +19,6 @@ RUN apt-get install -y --no-install-recommends \
     && apt-get clean
 
     {% endif %}
-{% elif install_type == 'source' %}
-    {% if base_distro in ['centos', 'fedora', 'oraclelinux', 'rhel'] %}
-
-# Install required packages
-RUN yum install -y openvswitch \
-    && yum clean all
-
-    {% endif %}
 {% endif %}
 
 {{ include_footer }}
diff --git a/docker/neutron/neutron-server/Dockerfile.j2 b/docker/neutron/neutron-server/Dockerfile.j2
index 9732e28f1..6540345db 100644
--- a/docker/neutron/neutron-server/Dockerfile.j2
+++ b/docker/neutron/neutron-server/Dockerfile.j2
@@ -9,3 +9,5 @@ RUN chmod 755 /usr/local/bin/kolla_extend_start \
        fi
 
 {{ include_footer }}
+
+USER neutron
diff --git a/docker/neutron/neutron-server/extend_start.sh b/docker/neutron/neutron-server/extend_start.sh
index d60829f10..dba1e632f 100644
--- a/docker/neutron/neutron-server/extend_start.sh
+++ b/docker/neutron/neutron-server/extend_start.sh
@@ -3,16 +3,6 @@
 # Bootstrap and exit if KOLLA_BOOTSTRAP variable is set. This catches all cases
 # of the KOLLA_BOOTSTRAP variable being set, including empty.
 if [[ "${!KOLLA_BOOTSTRAP[@]}" ]]; then
-    sudo -H -u neutron neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head
+    neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head
     exit 0
 fi
-
-# Neutron uses rootwrap which requires a tty for sudo.
-# Since the container is running in daemon mode, a tty
-# is not present and requiretty must be commented out.
-if [ ! -f /sudo-modified ]; then
-    chmod 0640 /etc/sudoers
-    sed -i '/Defaults    requiretty/s/^/#/' /etc/sudoers
-    chmod 0440 /etc/sudoers
-    touch /sudo-modified
-fi