Update TLS documentation

Updated TLS documentation to reflect new features and configuration
options added in Ussuri.

Change-Id: I74550eaf394287b14fc521293cc4b5ea8074192c
Partially-Implements: blueprint add-ssl-internal-network

1 files changed, 277 insertions, 84 deletions

diff --git a/doc/source/admin/tls.rst b/doc/source/admin/tls.rst
index eae7241fe..eefca8247 100644
--- a/doc/source/admin/tls.rst
+++ b/doc/source/admin/tls.rst
@@ -2,127 +2,320 @@
 TLS
 ===
 
-An additional endpoint configuration option is to enable or disable
-TLS protection for the internal and/or external VIP. TLS allows a client to
-authenticate the OpenStack service endpoint and allows for encryption of the
-requests and responses.
+This guide describes how to configure Kolla Ansible to deploy OpenStack with
+TLS enabled. Enabling TLS on the provided internal and/or external VIP address
+allows OpenStack clients to authenticate and encrypt network communication
+with OpenStack services.
 
-The configuration variables that control TLS networking are:
+When an OpenStack service exposes an API endpoint, Kolla Ansible will
+configure HAProxy for that service to listen on the internal and/or
+external VIP address. The HAProxy container load-balances requests on
+the VIPs to the nodes running the service container.
 
-- kolla_enable_tls_external
-- kolla_external_fqdn_cert
-- kolla_enable_tls_internal
-- kolla_internal_fqdn_cert
+There are two different layers of TLS configuration for OpenStack APIs:
+
+1. Enabling TLS on the internal and/or external VIP, so communication between
+   an OpenStack client and the HAProxy listening on the VIP is secure.
+2. Enabling TLS on the backend network, so communication between HAProxy and
+   the backend API services is secure.
 
 .. note::
 
-   If TLS is enabled only on the internal or the external network
-   the kolla_internal_vip_address and kolla_external_vip_address must
-   be different.
+  TLS authentication is based on certificates that have been signed by
+  trusted Certificate Authorities. Examples of commercial CAs are Comodo,
+  Symantec, GoDaddy, and GlobalSign. Letsencrypt.org is a CA that will
+  provide trusted certificates at no charge. If using a trusted CA is not
+  possible for your project, you can use a private CA, e.g. Hashicorp Vault,
+  to create a certificate for your domain, or see
+  :ref:`admin-tls-generating-a-private-ca` to use a Kolla Ansible generated
+  private CA.
+
+Quick Start
+~~~~~~~~~~~
 
-   If there is only a single network configured in your network topology
-   (opposed to configuring seperate internal and external networks), TLS
-   can be enabled using only the internal network configuration variables.
+.. note::
 
-The default for TLS is disabled, to enable TLS networking:
+  The certificates generated by Kolla Ansible use a simple Certificate
+  Authority setup and are not suitable for a production deployment. Only
+  certificates signed by a trusted Certificate Authority should be used in a
+  production deployment.
+
+To deploy OpenStack with TLS enabled for the external, internal and backend
+APIs, configure the following in ``globals.yml``:
 
 .. code-block:: yaml
 
-   kolla_enable_tls_external: "yes"
-   kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/mycert.pem"
+  kolla_enable_tls_internal: "yes"
+  kolla_enable_tls_external: "yes"
+  kolla_enable_tls_backend: "yes"
+  kolla_copy_ca_into_containers: "yes"
+
+If deploying on Debian or Ubuntu:
 
-   and/or
+.. code-block:: yaml
 
-   kolla_enable_tls_internal: "yes"
-   kolla_internal_fqdn_cert: "{{ kolla_certificates_dir }}/mycert-internal.pem"
+  openstack_cacert: "/etc/ssl/certs/ca-certificates.crt"
 
+If on CentOS or RHEL:
 
-.. note::
+.. code-block:: yaml
 
-   TLS authentication is based on certificates that have been
-   signed by trusted Certificate Authorities. Examples of commercial
-   CAs are Comodo, Symantec, GoDaddy, and GlobalSign. Letsencrypt.org
-   is a CA that will provide trusted certificates at no charge. Many
-   company's IT departments will provide certificates within that
-   company's domain. If using a trusted CA is not possible for your
-   situation, you can use `OpenSSL <https://www.openssl.org>`__
-   to create your own company's domain or see the section below about
-   kolla generated self-signed certificates.
-
-Two certificate files are required to use TLS securely with authentication.
-These two files will be provided by your Certificate Authority. These
-two files are the server certificate with private key and the CA certificate
-with any intermediate certificates. The server certificate needs to be
-installed with the kolla deployment and is configured with the
-``kolla_external_fqdn_cert`` or ``kolla_internal_fqdn_cert`` parameter.
-If the server certificate provided is not already trusted by the client,
-then the CA certificate file will need to be distributed to the client.
-
-When using TLS to connect to a public endpoint, an OpenStack client will
-have settings similar to this:
+  openstack_cacert: "/etc/pki/tls/certs/ca-bundle.crt"
 
-.. code-block:: shell
+The Kolla Ansible ``certificates`` command generates a private test Certificate
+Authority, and then uses the CA to sign the generated certificates for the
+enabled VIP(s) to test TLS in your OpenStack deployment. Assuming you are
+using the ``multinode`` inventory:
+
+.. code-block:: console
+
+  kolla-ansible -i ~/multinode certificates
+
+TLS Configuration for internal/external VIP
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-   export OS_PROJECT_DOMAIN_ID=default
-   export OS_USER_DOMAIN_ID=default
-   export OS_PROJECT_NAME=demo
-   export OS_USERNAME=demo
-   export OS_PASSWORD=demo-password
-   export OS_AUTH_URL=https://mykolla.example.net:5000
-   # os_cacert is optional for trusted certificates
-   export OS_CACERT=/etc/pki/ca/mykolla-cacert.crt
-   export OS_IDENTITY_API_VERSION=3
+The configuration variables that control TLS for the internal and/or external
+VIP are:
 
-Self-Signed Certificates
-~~~~~~~~~~~~~~~~~~~~~~~~
+- ``kolla_enable_tls_external``
+- ``kolla_enable_tls_internal``
+- ``kolla_internal_fqdn_cert``
+- ``kolla_external_fqdn_cert``
 
 .. note::
 
-   Self-signed certificates should never be used in production.
+  If TLS is enabled only on the internal or external network, then
+  ``kolla_internal_vip_address`` and ``kolla_external_vip_address`` must
+  be different.
 
-It is not always practical to get a certificate signed by a well-known
-trust CA, for example a development or internal test kolla deployment. In
-these cases it can be useful to have a self-signed certificate to use.
+  If there is only a single network configured in your topology
+  (as opposed to separate internal and external networks), TLS
+  can only be enabled using the internal network configuration variables.
 
-For convenience, the ``kolla-ansible`` command will generate the necessary
-certificate files based on the information in the ``globals.yml``
-configuration file:
+The default state for TLS networking is disabled. To enable external TLS
+encryption:
 
-.. code-block:: console
+.. code-block:: yaml
+
+  kolla_enable_tls_external: "yes"
+
+To enable internal TLS encryption:
+
+.. code-block:: yaml
+
+  kolla_enable_tls_internal: "yes"
+
+Two certificate files are required to use TLS securely with authentication,
+which will be provided by your Certificate Authority:
+
+* server certificate with private key
+* CA certificate with any intermediate certificates
+
+The combined server certificate and private key needs to be provided to Kolla
+Ansible, with the path configured via ``kolla_external_fqdn_cert`` or
+``kolla_internal_fqdn_cert``.  These paths default to ``{{
+kolla_certificates_dir }}/haproxy.pem`` and ``{{ kolla_certificates_dir
+}}/haproxy-internal.pem`` respectively, where ``kolla_certificates_dir`` is
+``/etc/kolla/certificates`` by default.
+
+If the server certificate provided is not already trusted by clients, then
+the CA certificate file will need to be distributed to the clients. This is
+discussed in more detail in :ref:`admin-tls-openrc` and
+:ref:`admin-tls-ca-in-containers`.
+
+.. _admin-tls-openrc:
+
+Configuring the OpenStack Client for TLS
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The location for the CA certificate for the ``admin-openrc.sh`` file is
+configured with the ``kolla_admin_openrc_cacert`` variable, which is not set by
+default. This must be a valid path on all hosts where ``admin-openrc.sh`` is
+used.
+
+When TLS is enabled on a VIP, and ``kolla_admin_openrc_cacert`` is set to
+``/etc/pki/tls/certs/ca-bundle.crt``, an OpenStack client will have settings
+similar to this configured by ``admin-openrc.sh``:
 
-   kolla-ansible certificates
+.. code-block:: shell
 
-The certificate file haproxy.pem will be generated and stored in the
-``/etc/kolla/certificates/`` directory, and the CA cert will be in the
-``/etc/kolla/certificates/ca/`` directory.
+  export OS_PROJECT_DOMAIN_NAME=Default
+  export OS_USER_DOMAIN_NAME=Default
+  export OS_PROJECT_NAME=admin
+  export OS_TENANT_NAME=admin
+  export OS_USERNAME=admin
+  export OS_PASSWORD=demoPassword
+  export OS_AUTH_URL=https://mykolla.example.net:5000
+  export OS_INTERFACE=internal
+  export OS_ENDPOINT_TYPE=internalURL
+  export OS_MISTRAL_ENDPOINT_TYPE=internalURL
+  export OS_IDENTITY_API_VERSION=3
+  export OS_REGION_NAME=RegionOne
+  export OS_AUTH_PLUGIN=password
+  # os_cacert is optional for trusted certificates
+  export OS_CACERT=/etc/pki/tls/certs/ca-bundle.crt
+
+.. _admin-tls-ca-in-containers:
 
 Adding CA Certificates to the Service Containers
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To copy CA certificate files to the service containers
+To copy CA certificate files to the service containers:
 
 .. code-block:: yaml
 
-   kolla_copy_ca_into_containers: "yes"
+  kolla_copy_ca_into_containers: "yes"
 
-When ``kolla_copy_ca_into_containers`` is configured to "yes", the
-CA certificate files in /etc/kolla/certificates/ca will be copied into
+When ``kolla_copy_ca_into_containers`` is configured to "yes", the CA
+certificate files in ``/etc/kolla/certificates/ca`` will be copied into
 service containers to enable trust for those CA certificates. This is required
 for any certificates that are either self-signed or signed by a private CA,
-and are not already present in the service image trust store.
+and are not already present in the service image trust store. Kolla will
+install these certificates in the container system wide trust store when the
+container starts.
+
+All certificate file names will have the ``kolla-customca-`` prefix prepended
+to them when they are copied into the containers. For example, if a certificate
+file is named ``internal.crt``, it will be named
+``kolla-customca-internal.crt`` in the containers.
+
+For Debian and Ubuntu containers, the certificate files will be copied to the
+``/usr/local/share/ca-certificates/`` directory.
+
+For CentOS and RHEL containers, the certificate files will be copied to the
+``/etc/pki/ca-trust/source/anchors/`` directory.
+
+In both cases, valid certificates will be added to the system trust store -
+``/etc/ssl/certs/ca-certificates.crt`` on Debian and Ubuntu, and
+``/etc/pki/tls/certs/ca-bundle.crt`` on CentOS and RHEL.
+
+Configuring a CA bundle
+~~~~~~~~~~~~~~~~~~~~~~~
+
+OpenStack services do not always trust CA certificates from the system trust
+store by default. To resolve this, the ``openstack_cacert`` variable should be
+configured with the path to the CA Certificate in the container.
+
+To use the system trust store on Debian or Ubuntu:
+
+.. code-block:: yaml
+
+   openstack_cacert: /etc/ssl/certs/ca-certificates.crt
+
+For CentOS or RHEL:
+
+.. code-block:: yaml
+
+   openstack_cacert: /etc/pki/tls/certs/ca-bundle.crt
+
+Back-end TLS Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Enabling TLS on the backend services secures communication between the
+HAProxy listing on the internal/external VIP and the OpenStack
+services. It also enables secure end-to-end communication between OpenStack
+services that support TLS termination. The OpenStack services that support
+backend TLS termination in Ussuri are: Keystone, Glance, Heat, Placement,
+Horizon, Barbican, and Cinder.
+
+The configuration variables that control back-end TLS for service endpoints
+are:
+
+- ``kolla_enable_tls_backend``
+- ``kolla_tls_backend_cert``
+- ``kolla_tls_backend_key``
+- ``haproxy_backend_cacert``
+- ``haproxy_backend_cacert_dir``
+
+The default state for back-end TLS is disabled. To enable TLS for the back-end
+communication:
+
+.. code-block:: yaml
+
+  kolla_enable_tls_backend: "yes"
+
+It is also possible to enable back-end TLS on a per-service basis. For example,
+to enable back-end TLS for Keystone, set ``keystone_enable_tls_backend`` to
+``yes``.
+
+The default values for ``haproxy_backend_cacert`` and
+``haproxy_backend_cacert_dir`` should suffice if the certificate is in the
+system trust store. Otherwise, they should be configured to a location of the
+CA certificate installed in the service containers.
+
+Each backend service requires a certificate and private key. In many cases it
+is necessary to use a separate certificate and key for each host, or even
+per-service. The following precedence is used for the certificate:
+
+* ``{{ kolla_certificates_dir }}/{{ inventory_hostname }}/{{ project_name
+  }}-cert.pem``
+* ``{{ kolla_certificates_dir }}/{{ inventory_hostname }}-cert.pem``
+* ``{{ kolla_certificates_dir }}/{{ project_name }}-cert.pem``
+* ``{{ kolla_tls_backend_cert }}``
+
+And for the private key:
+
+* ``{{ kolla_certificates_dir }}/{{ inventory_hostname }}/{{ project_name
+  }}-key.pem``
+* ``{{ kolla_certificates_dir }}/{{ inventory_hostname }}-key.pem``
+* ``{{ kolla_certificates_dir }}/{{ project_name }}-key.pem``
+* ``{{ kolla_tls_backend_key }}``
+
+The default for ``kolla_certificates_dir`` is ``/etc/kolla/certificates``.
+
+``kolla_tls_backend_cert`` and ``kolla_tls_backend_key``, default to ``{{
+kolla_certificates_dir }}/backend-cert.pem`` and ``{{ kolla_certificates_dir
+}}/backend-key.pem`` respectively.
+
+``project_name`` is the name of the OpenStack service, e.g. ``keystone`` or
+``cinder``.
+
+.. note::
+  The back-end TLS cert/key can be the same certificate that is used for the
+  VIP, as long as those certificates are configured to allow requests from
+  both the VIP and internal networks.
+
+By default, the TLS certificate will be verified as trustable by the
+OpenStack services. Although not recommended for production, it is possible to
+disable verification of the backend certificate:
+
+.. code-block:: yaml
+
+  kolla_verify_tls_backend: "no"
+
+.. _admin-tls-generating-a-private-ca:
+
+Generating a Private Certificate Authority
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. note::
+
+  The certificates generated by Kolla Ansible use a simple Certificate
+  Authority setup and are not suitable for a production deployment. Only
+  certificates signed by a trusted Certificate Authority should be used in a
+  production deployment.
+
+It's not always practical to get a certificate signed by a trusted CA. In a
+development or internal test OpenStack deployment, it can be useful to
+generate certificates locally to enable TLS.
+
+For convenience, the ``kolla-ansible`` command will generate the necessary
+certificate files based on the information in the ``globals.yml``
+configuration file and the inventory file:
+
+.. code-block:: console
 
-All certificate file names will have the "kolla-customca-" prefix prepended to
-it when it is copied into the containers. For example, if a certificate file is
-named "internal.crt", it will be named "kolla-customca-internal.crt" in the
-containers.
+  kolla-ansible -i multinode certificates
 
-For Debian and Ubuntu containers, the certificate files will be copied to
-the ``/usr/local/share/ca-certificates/`` directory.
+The ``certificates`` role performs the following actions:
 
-For Centos and Red Hat Linux containers, the certificate files will be copied
-to the ``/etc/pki/ca-trust/source/anchors/`` directory.
+1. Generates a test root Certificate Authority
+2. Generates the internal/external certificates which are signed by the root
+   CA.
+3. If back-end TLS is enabled, generate the back-end certificate signed by
+   the root CA.
 
-In addition, the ``openstack_cacert`` should be configured with the path to
-the cacert in the container. For example, if the self-signed certificate task
-was used and the deployment is on ubuntu, the path would be:
-"/etc/pki/ca-trust/source/anchors/kolla-customca-haproxy-internal.crt"
+The combined certificate and key file ``haproxy.pem`` (which is the default
+value for ``kolla_external_fqdn_cert``) will be generated and stored in the
+``/etc/kolla/certificates/`` directory, and a copy of the CA certificate
+(``root.crt``) will be stored in the ``/etc/kolla/certificates/ca/`` directory.