如何在OpenStack-Ansible上集成Tungsten Fabric

本文作者James Denton，拥有超过15年信息技术领域行业经验，目前为知名云计算服务商Rackspace Hosting首席架构师，致力于云网络和对基于OpenStack的Rackspace私有云产品的支持。

Tungsten Fabric（前身为OpenContrail）是Linux基金会旗下的“多云、多技术栈”SDN解决方案。简而言之，Tungsten Fabric和Contrail（基于TF的商业产品）可以替代和增强标准OpenStack云的许多网络组件，并提供以下功能：

·分布式虚拟路由

·DHCP和元数据服务

·基于策略的访问控制

·与安全组的兼容性

·……等等

转发平面支持MPLS over GRE、VXLAN、L2/L3单播和L3多播，以实现虚拟网络和物理网络之间的互连。

注：关于Tungsten Fabric架构的概述可在下面的链接找到：https://tungstenfabric.org.cn/docs

我最近接受了将Tungsten Fabric集成到OpenStack-Ansible中的挑战，以简化TF和相关OpenStack bits在生产级OpenStack云中的部署。本文将在较高级别涵盖OpenStack-Ansible的主（Rocky）分支的一些补丁，以及Juniper和TF社区提供的用于部署Tungsten Fabric的contrail-ansible-deployer playbooks的一些调整。本文所描述的过程绝不意味着是最终的过程，可能会显得比较笨拙且不理想，但是，这是一个开始。

注：本文作者指明文章中的Tungsten Fabric、OpenContrail和Contrail为混用，翻译时统一为Tungsten Fabric。

集成要求

几周前，我部署了一个独立的3节点Tungsten Fabric设置，其中包含基于Queens的OpenStack-Ansible的All-in-One节点。在弄清楚使事情进入半工作状态所需的调整后，我决定亲自尝试并部署一个AIO节点，该节点包含一个单一的Tungsten Fabric服务实例以及基本的OpenStack服务。

以下为最低规格的配置建议：

·系统: Ubuntu VM

·OS: 16.04.4 LTS

·内存: 48GB

·硬盘: 300GB

·NIC: 单口网卡

如我稍后指出的，裸机节点可能复杂度较小，但是ESXi或其它管理程序上的虚拟机应该是可以尝试的。

从OpenStack-Ansible开始

首先，请克隆OpenStack-Ansible存储库。在撰写本文时，master分支是与OpenStack的第18个发行版Rocky相关联的。

代码语言：javascript复制

# git clone https://git.openstack.org/openstack/openstack-ansible /opt/openstack-ansible
# cd /opt/openstack-ansible
# git checkout master
# export ANSIBLE_ROLE_FETCH_MODE=git-clone

接下来，运行引导程序脚本：

代码语言：javascript复制

# scripts/bootstrap-ansible.sh
# scripts/bootstrap-aio.sh

引导程序脚本将下载playbooks以部署OpenStack，还将在服务器上准备符合OpenStack-Ansible架构的网络环境。

角色修改

对使用OpenStack-Ansible部署的OpenStack云进行更改，通常意味着对构成部署的Ansible角色进行更改。这包括对任务、模板、变量等的更改。

需要修改的角色包括：

·os_neutron

·os_nova

是否所有这些角色的更改都是必需的，还有待观察，这里只是为了更好地说明。

os_neutron

一些新文件包括：

代码语言：javascript复制

root@aio1:/etc/ansible/roles/os_neutron# git diff --staged
diff --git a/tasks/providers/opencontrail_config.yml b/tasks/providers/opencontrail_config.yml
new file mode 100644
index 0000000..8f5fc7d
--- /dev/null
    b/tasks/providers/opencontrail_config.yml
@@ -0,0  1,99 @@
 ---
 # Copyright 2018, Rackspace Hosting, Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 # http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
 - name: Set the packages to install
  set_fact:
  neutron_optional_combined_pip_packages: |-
  {% set packages = neutron_optional_opencontrail_pip_packages %}
  {{ packages }}
 
 - name: Install OpenContrail pip packages
  pip:
  name: "{{ neutron_optional_combined_pip_packages }}"
  state: "{{ neutron_pip_package_state }}"
  virtualenv: "{{ neutron_bin | dirname }}"
  virtualenv_site_packages: "no"
  extra_args: >-
  {{ neutron_developer_mode | ternary(pip_install_developer_constraints | default('--constraint /opt/developer-pip-constraints.txt'), '') }}
  {{ (pip_install_upper_constraints is defined) | ternary('--constraint '   pip_install_upper_constraints | default(''),'') }}
  {{ pip_install_options | default('') }}
  register: install_packages
  until: install_packages|success
  retries: 5
  delay: 2
  tags:
  - opencontrail-install
  - opencontrail-pip-packages
 
 - name: Install git
  apt:
  name: git
  state: present
  delegate_to: "{{ item }}"
  with_items:
  - "{{ groups['neutron_server'] }}"
  tags:
  - opencontrail-install
 
 - name: Clone contrail neutron plugin
  git:
  repo: "{{ opencontrail_plugin_git_repo }}"
  version: "{{ opencontrail_plugin_git_install_branch }}"
  dest: /opt/contrail-neutron-plugin
  force: yes
  register: contrail_plugin_git_clone
  delegate_to: "{{ item }}"
  with_items:
  - "{{ groups['neutron_server'] }}"
  until: contrail_plugin_git_clone|success
  retries: 5
  delay: 2
  tags:
  - opencontrail-install
 
 # (jamesdenton) Will need to eventually compile and/or extract from Docker container
 # The tasks marked (temp) should be reworked
 
 - name: Download Contrail python libraries (temp)
  vars:
  - dlpath: https://github.com/busterswt/contrail-openstack/raw/master
  get_url:
  url: "{{ dlpath }}/{{ item }}"
  dest: /opt
  mode: 0440
  with_items:
  - contrail-openstack-neutron-init.tar
  tags:
  - opencontrail-install
 
 - name: Unpack Contrail python libraries (temp)
  unarchive:
  remote_src: yes
  src: /opt/contrail-openstack-neutron-init.tar
  dest: /openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages
  when:
  inventory_hostname == groups['neutron_server'][0]
  tags:
  - opencontrail-install
 
 - name: Install contrail neutron plugin into venv
  command: "/openstack/venvs/neutron-{{ neutron_venv_tag }}/bin/python setup.py install"
  args:
  chdir: /opt/contrail-neutron-plugin
  delegate_to: "{{ item }}"
  with_items:
  - "{{ groups['neutron_server'] }}"
  tags:
  - opencontrail-install
diff --git a/templates/plugins/opencontrail/ContrailPlugin.ini.j2 b/templates/plugins/opencontrail/ContrailPlugin.ini.j2
new file mode 100644
index 0000000..9d645b0
--- /dev/null
    b/templates/plugins/opencontrail/ContrailPlugin.ini.j2
@@ -0,0  1,23 @@
 # {{ ansible_managed }}
 
 {% if neutron_plugin_type == 'opencontrail' %}
 [APISERVER]
 api_server_ip = {{ opencontrail_api_vip_address }}
 api_server_port = {{ opencontrail_api_vip_port }}
 multi_tenancy = True
 contrail_extensions = ipam:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_ipam.NeutronPluginContrailIpam,policy:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_policy.NeutronPluginContrailPolicy,route-table:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_vpc.NeutronPluginContrailVpc,contrail:None,service-interface:None,vf-binding:None
 
 [COLLECTOR]
 analytics_api_ip = {{ opencontrail_collector_vip_address }}
 analytics_api_port = {{ opencontrail_collector_vip_port }}
 
 [keystone_authtoken]
 auth_host = {{ internal_lb_vip_address }}
 auth_port = {{ keystone_service_port }}
 auth_protocol = {{ keystone_service_proto }}
 admin_user = {{ keystone_admin_user_name }}
 admin_password = {{ keystone_auth_admin_password }}
 admin_tenant_name = {{ keystone_admin_tenant_name }}
 insecure = True
 region_name = {{ keystone_service_region }}
 {% endif %}

对现有文件的更改包括：

代码语言：javascript复制

root@aio1:/etc/ansible/roles/os_neutron# git diff
diff --git a/defaults/main.yml b/defaults/main.yml
index 162e933..7054c96 100644
--- a/defaults/main.yml
    b/defaults/main.yml
@@ -63,6  63,8 @@ networking_bgpvpn_git_repo: https://git.openstack.org/openstack/networking-bgpvp
 networking_bgpvpn_git_install_branch: master
 openstack_ceilometer_git_repo: https://git.openstack.org/openstack/ceilometer
 openstack_ceilometer_git_install_branch: master
 opencontrail_plugin_git_repo: https://github.com/Juniper/contrail-neutron-plugin
 opencontrail_plugin_git_install_branch: master
 # Developer mode
 neutron_developer_mode: false
@@ -164,6  166,7 @@ neutron_sriov_nic_agent_ini_overrides: {}
 neutron_sriov_nic_agent_init_overrides: {}
 neutron_vpn_agent_init_overrides: {}
 neutron_vpnaas_agent_ini_overrides: {}
 neutron_opencontrail_conf_ini_overrides: {}
 ###
 ### Quotas
@@ -434,3  437,12 @@ ovs_nsh_support: False
 # Set higher priority to mardim PPA when ovs_nsh_support is True
 ovs_nsh_apt_pinned_packages: [{ package: "*", release: "LP-PPA-mardim-mardim-ppa"}]
 
 ###
 ### Contrail/OpenContrail/Tungsten Fabric Configuration
 ###
 
 opencontrail_api_vip_address: "{{ external_lb_vip_address }}"
 opencontrail_api_vip_port: "8082"
 opencontrail_collector_vip_address: "{{ external_lb_vip_address }}"
 opencontrail_collector_vip_port: "8081"
diff --git a/templates/neutron.conf.j2 b/templates/neutron.conf.j2
index 83d25a7..dd755ca 100644
--- a/templates/neutron.conf.j2
    b/templates/neutron.conf.j2
@@ -42,6  42,10 @@ core_plugin = {{ neutron_plugin_core }}
 {% if neutron_plugin_type.split('.')[0] == 'ml2' %}
 service_plugins = {{ neutron_plugin_loaded_base | join(',') }}
 {% endif %}
 {% if neutron_plugin_type == 'opencontrail' %}
 service_plugins = neutron_plugin_contrail.plugins.opencontrail.loadbalancer.v2.plugin.LoadBalancerPluginV2
 api_extensions_path = /openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages/neutron_plugin_contrail/extensions:/openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages/neutron_lbaas/extensions
 {% endif %}
 # MAC address generation for VIFs
 base_mac = fa:16:3e:00:00:00
@@ -94,8  98,9 @@ rpc_workers = {{ neutron_rpc_workers }}
 {% set dhcp_agents_max = num_agent if num_agent > 2 else 2 %}
 # DHCP
-{% if neutron_plugin_type == 'ml2.dragonflow' %}
-# In dragonflow, DHCP is fully distributed, and DHCP agents are not used
 {% if neutron_plugin_type == ('ml2.dragonflow' or 'opencontrail') %}
 # In dragonflow and opencontrail, DHCP is fully distributed and DHCP
 # agents are not used
 dhcp_agent_notification = False
 {% else %}
 dhcp_agent_notification = True
diff --git a/vars/main.yml b/vars/main.yml
index cef4ee8..2d1c2a2 100644
--- a/vars/main.yml
    b/vars/main.yml
@@ -121,6  121,10 @@ neutron_plugins:
 plugin_ini: plugins/ml2/ml2_conf.ini
 driver_interface: "openvswitch"
 l3_agent_mode: "legacy"
  opencontrail:
  plugin_core: neutron_plugin_contrail.plugins.opencontrail.contrail_plugin.NeutronPluginContrailCoreV2
  plugin_ini: plugins/opencontrail/ContrailPlugin.ini
  plugin_conf_ini_overrides: "{{ neutron_opencontrail_conf_ini_overrides }}"
 ###
 ### ML2 Plugin Configuration
diff --git a/vars/source_install.yml b/vars/source_install.yml
index a246a45..24e57ea 100644
--- a/vars/source_install.yml
    b/vars/source_install.yml
@@ -96,6  96,13 @@ neutron_proprietary_nuage_pip_packages:
 - nuage-openstack-neutronclient
 - nuagenetlib
 neutron_optional_opencontrail_pip_packages:
  - bitarray
  - bottle
  - geventhttpclient
  - psutil>=0.6.0
  - requests>=1.1.0
 
 neutron_developer_constraints:
 - "git {{ neutron_git_repo }}@{{ neutron_git_install_branch }}#egg=neutron"
 - "git {{ neutron_fwaas_git_repo }}@{{ neutron_fwaas_git_install_branch }}#egg=neutron-fwaas"

os_nova

代码语言：javascript复制

root@aio1:/etc/ansible/roles/os_nova# git diff
diff --git a/defaults/main.yml b/defaults/main.yml
index 67d92e9..bc44511 100644
--- a/defaults/main.yml
    b/defaults/main.yml
@@ -325,6  325,9 @@ nova_network_services:
 calico:
 use_forwarded_for: True
 metadata_proxy_enabled: False
  opencontrail:
  use_forwarded_for: True
  metadata_proxy_enabled: True
 # Nova quota
 nova_quota_cores: 20

HAproxy更改

为了提供单一的TF API和仪表板终端，我决定创建一个VIP，以平衡TF API和analytics服务之间的流量。这是否为最佳做法还有待观察，但是此处对group_vars的更改有助于创建VIP：

代码语言：javascript复制

root@aio1:/opt/openstack-ansible# git diff
diff --git a/inventory/group_vars/haproxy/haproxy.yml b/inventory/group_vars/haproxy/haproxy.yml
index b837443..dc53ef4 100644
--- a/inventory/group_vars/haproxy/haproxy.yml
    b/inventory/group_vars/haproxy/haproxy.yml
@@ -36,6  36,7 @@ haproxy_rabbitmq_management_whitelist_networks: "{{ haproxy_whitelist_networks }
 haproxy_repo_git_whitelist_networks: "{{ haproxy_whitelist_networks }}"
 haproxy_repo_cache_whitelist_networks: "{{ haproxy_whitelist_networks }}"
 haproxy_opendaylight_whitelist_networks: "{{ haproxy_whitelist_networks }}"
 haproxy_opencontrail_whitelist_networks: "{{ haproxy_whitelist_networks }}"
 haproxy_default_services:
 - service:
@@ -365,3  366,23 @@ haproxy_default_services:
 haproxy_backend_httpcheck_options:
 - expect status 405
 haproxy_service_enabled: "{{ (groups['ceph-rgw'] is defined and groups['ceph-rgw'] | length > 0) or (ceph_rgws | length > 0) }}"
  - service:
  haproxy_service_name: opencontrail-api
  haproxy_backend_nodes: "{{ groups['opencontrail-api_hosts'] | default([]) }}"
  haproxy_bind: "{{ [opencontrail_api_vip_address] }}"
  haproxy_port: 8082
  haproxy_balance_type: tcp
  haproxy_timeout_client: 5000s
  haproxy_timeout_server: 5000s
  haproxy_whitelist_networks: "{{ haproxy_opencontrail_whitelist_networks }}"
  haproxy_service_enabled: "{{ neutron_plugin_type == 'opencontrail' }}"
  - service:
  haproxy_service_name: opencontrail-collector
  haproxy_backend_nodes: "{{ groups['opencontrail-analytics_hosts'] | default([]) }}"
  haproxy_bind: "{{ [opencontrail_collector_vip_address] }}"
  haproxy_port: 8081
  haproxy_balance_type: tcp
  haproxy_timeout_client: 5000s
  haproxy_timeout_server: 5000s
  haproxy_whitelist_networks: "{{ haproxy_opencontrail_whitelist_networks }}"
  haproxy_service_enabled: "{{ neutron_plugin_type == 'opencontrail' }}"

我在All-in-One节点上使用此方法遇到的一些问题，包括HAproxy将VIP绑定到端口8081的能力。后来在过程中发现，Tungsten Fabric playbooks在0.0.0.0:8081上创建了一个侦听器，从而阻止了VIP绑定在同一端口上。这里的替代方法是注释掉该服务，或在部署HAproxy后将其禁用。对于HAproxy在不同节点上的多节点安装，可以保留它。最终，这两种服务的负载平衡可能无法实现，但我暂时将其保留。

环境更改

默认的Neutron env.d框架将部署原本没必要部署的Neutron代理容器。通过覆盖默认值，我们可以删除代理容器并定义一些新的组件：

代码语言：javascript复制

cat <<'EOF' >> /etc/openstack_deploy/env.d/neutron.yml
component_skel:
 neutron_server:
 belongs_to:
 - neutron_all
 opencontrail_vrouter:
 belongs_to:
 - neutron_all
 opencontrail_api:
 belongs_to:
 - neutron_all
 opencontrail_analytics:
 belongs_to:
 - neutron_all
container_skel:
 neutron_agents_container:
 contains: {}
 opencontail-vrouter_container:
 belongs_to:
 - compute_containers
 contains:
 - opencontrail_vrouter
 properties:
 is_metal: true
 opencontrail-api_container:
 belongs_to:
 - opencontrail-api_containers
 contains:
 - opencontrail_api
 properties:
 is_metal: true
 opencontrail-analytics_container:
 belongs_to:
 - opencontrail-analytics_containers
 contains:
 - opencontrail_analytics
 properties:
 is_metal: true
 neutron_server_container:
 belongs_to:
 - network_containers
 contains:
 - neutron_server
physical_skel:
 opencontrail-api_containers:
 belongs_to:
 - all_containers
 opencontrail-api_hosts:
 belongs_to:
 - hosts
 opencontrail-analytics_containers:
 belongs_to:
 - all_containers
 opencontrail-analytics_hosts:
 belongs_to:
 - hosts
EOF

在这一点上，我不确定所有这些更改都是必要的，以及方法是否正确，但目前暂时这样。

在 openstack_user_config.yml文件中，我定义了两个新的组，以便能够在主机之间拆分API和Analytics服务。由于这是一个AlO节点，因此IP是相同的：

代码语言：javascript复制

opencontrail-api_hosts:
 aio1:
 ip: 172.29.236.100
opencontrail-analytics_hosts:
 aio1:
 ip: 172.29.236.100

Overrides

对os_neutron角色的更改导致添加了新的默认值，并且还需要一些overrides。在user_variables.yml中添加了以下内容：

代码语言：javascript复制

neutron_plugin_type: opencontrail
neutron_driver_quota: neutron_plugin_contrail.plugins.opencontrail.quota.driver.QuotaDriver
opencontrail_plugin_git_install_branch: R5.0

角色中指定的一些默认值包括：

代码语言：javascript复制

opencontrail_api_vip_address: "{{ external_lb_vip_address }}"
opencontrail_api_vip_port: "8082"
opencontrail_collector_vip_address: "{{ external_lb_vip_address }}"
opencontrail_collector_vip_port: "8081"
opencontrail_plugin_git_repo: https://github.com/Juniper/contrail-neutron-plugin
opencontrail_plugin_git_install_branch: master
neutron_opencontrail_conf_ini_overrides: {}

最终要求仍尚未确定。

运行OpenStack-Ansible playbooks

至此，已经完成了对OpenStack-Ansible playbooks的大部分更改，并且可以部署OpenStack了：

代码语言：javascript复制

# cd /opt/openstack-ansible/playbooks
# openstack-ansible setup-hosts.yml
# openstack-ansible setup-infrastructure.yml
# openstack-ansible setup-openstack.yml

虽然Neutron会被下线，但在使用网络之前，还需要进行一些其它更改并部署Tungsten Fabric。可以使用Horizon，但可能无法完全发挥它的功能。

接下来，我们必须要执行一些操作，包括克隆Juniper ansible playbook repo，设置overrides，以及运行这些playbooks，以安装Tungsten Fabric，并为与TF相关的服务overlay一些Docker容器。

部署Tungsten Fabric

到这里，我们已经部署了OpenStack，并且知道已实现的网络后端，但是该后端还不存在。Juniper提供的playbooks可以使用Docker容器部署Tungsten Fabric。这些相同的playbooks还可以部署基于Kolla的OpenStack版本，这是另一种OpenStack部署策略。在这里，我们实际上只需要与Tungsten Fabric相关的代码。

为了实现一些更改，我克隆了repo，并进行了部署在基于OpenStack-Ansible的云上所需的必要更改。

克隆repo：

代码语言：javascript复制

# git clone http://github.com/busterswt/contrail-ansible-deployer /opt/openstack-ansible/playbooks/contrail-ansible-deployer
# cd /opt/openstack-ansible/playbooks/contrail-ansible-deployer
# git checkout osa

Overrides

Juniper playbooks依赖于它们自己的清单和overrides，因此执行以下几步可能会感觉有些多余。我还没有重新编写playbooks以利用OpenStack-Ansible的可用清单。

可以在/etc/openstack_deploy/user_opencontrail_vars.yml的新文件中定义这些overrides：

代码语言：javascript复制

config_file: /etc/openstack_deploy/user_opencontrail_vars.yml
opencontrail_api_vip_address: ""
opencontrail_collector_vip_address: ""
provider_config:
 bms:
 ssh_pwd:
 ssh_user: root
 ssh_public_key: /root/.ssh/id_rsa.pub
 ssh_private_key: /root/.ssh/id_rsa
 ntpserver: 129.6.15.28
instances:
 aio1:
 provider: bms
 ip: 172.29.236.100
 roles:
 config_database:
 config:
 control:
 analytics_database:
 analytics:
 webui:
 vrouter:
  VROUTER_GATEWAY: 10.50.0.1
  PHYSICAL_INTERFACE: ens160
global_configuration:
 CONTAINER_REGISTRY: opencontrailnightly
# CONTAINER_REGISTRY: hub.juniper.net/contrail
# CONTAINER_REGISTRY_USERNAME: 
# CONTAINER_REGISTRY_PASSWORD: 
contrail_configuration:
 CLOUD_ORCHESTRATOR: openstack
 CONTRAIL_VERSION: latest
# CONTRAIL_VERSION: 5.0.0-0.40
# UPGRADE_KERNEL: true
 UPGRADE_KERNEL: false
 KEYSTONE_AUTH_HOST: ""
 KEYSTONE_AUTH_PUBLIC_PORT: 5000
 KEYSTONE_AUTH_PROTO: http
 KEYSTONE_AUTH_URL_VERSION:
 KEYSTONE_AUTH_ADMIN_USER: admin
 KEYSTONE_AUTH_ADMIN_PASSWORD: ""
 KEYSTONE_AUTH_URL_VERSION: /v3
 CONTROLLER_NODES: 172.29.236.100
 CONTROL_NODES: 172.29.236.100
 ANALYTICSDB_NODES: 172.29.236.100
 WEBUI_NODES: 172.29.236.100
 ANALYTICS_NODES: 172.29.236.100
 CONFIGDB_NODES: 172.29.236.100
 CONFIG_NODES: 172.29.236.100

请特别注意以下键/值对：

代码语言：javascript复制

VROUTER_GATEWAY: 10.50.0.1
PHYSICAL_INTERFACE: ens160
CONTAINER_REGISTRY: opencontrailnightly
# CONTAINER_REGISTRY: hub.juniper.net/contrail
# CONTAINER_REGISTRY_USERNAME: 
# CONTAINER_REGISTRY_PASSWORD: 
CONTRAIL_VERSION: latest
# CONTRAIL_VERSION: 5.0.0-0.40
CONTROLLER_NODES: 172.29.236.100
CONTROL_NODES: 172.29.236.100
ANALYTICSDB_NODES: 172.29.236.100
WEBUI_NODES: 172.29.236.100
ANALYTICS_NODES: 172.29.236.100
CONFIGDB_NODES: 172.29.236.100
CONFIG_NODES: 172.29.236.100

第一部分定义了vRouter所需的vhost0 接口将重新利用或者利用的“physical”接口。这是一个具有单个NIC的AIO节点，这意味着机会很不错，playbooks会自动确定要使用的接口。我们继续前进，同时在此处进行了标注。该主机的IP地址是10.50.0.221，网关是10.50.0.1。对于多宿主主机，有可能将接口及其自己的网关地址专用于vRouter。我尚未进行多NIC部署，但期待能实现。

第二部分定义了将从中下载容器的Docker注册表。这里opencontailnightly 注册表包含每隔一到两天生成的容器，这些容器不一定在任何给定的日期都是有效的。如果你通过Juniper访问GA注册表，还可以定义该注册表并提供访问凭据。在nightly注册表中唯一可用的版本是latest版本，而Juniper注册表可能具有已标记的发行版。记得要使用适当的注册表。

第三部分定义了各个节点类型的地址，用于在整个playbooks中用于将值插入模板。这些是当前成功完成playbooks所必需的。请注意，我在这里使用了“CONTAINER_NET”地址，是希望Tungsten Fabric和OpenStack可以在LXC使用的现有容器网络上进行通信。在此AIO节点中，172.29.236.100是br-mgmt上配置的IP，并且恰好也是内部VIP地址。部署在Docker容器中的服务会将端口绑定到其各自服务的IP上。在多节点的安装中，此更改的方式是否可行仍为待定。

OSA repo更改

OpenStack-Ansible包括用于软件包管理repo服务器，该服务器目前禁止安装部署Tungsten Fabric所需的某些版本的软件包。在这里，我通过修改/root/.pip/pip.conf禁用了内部repo服务器：

代码语言：javascript复制

[global]
disable-pip-version-check = true
timeout = 120
#index-url = http://172.29.236.100:8181/simple
#trusted-host =
# 172.29.236.100
[install]
upgrade = true
upgrade-strategy = only-if-needed
pre = true

运行playbooks

到这里，我们应该有了一个很好的起点，可以开始运行TF playbooks了。

首先，节点必须运行引导程序：

代码语言：javascript复制

# cd /opt/openstack-ansible/playbooks/
# openstack-ansible -e orchestrator=openstack contrail-ansible-deployer/playbooks/configure_instances.yml

当UPGRADE_KERNEL 为true时，主机可能会重新启动。如果发生这种情况，请再次重新运行该playbook。如果TF节点不是需要部署的主机，则playbooks具有一个计时器，等待主机返回。

接下来，运行install_contrail.yml playbook以部署TF：

代码语言：javascript复制

# cd /opt/openstack-ansible/playbooks/
# openstack-ansible -e orchestrator=openstack contrail-ansible-deployer/playbooks/install_contrail.yml

安装其它组件

由于Juniper playbooks期望基于Kolla的部署，因此某些组件未安装在基于OpenStack-Ansible的云之上。我自己从Docker容器中提取了一些实用程序、模块，以及更多的内容，并编写了一个playbook来实现这些内容：

代码语言：javascript复制

# cd /opt/openstack-ansible/playbooks/
# openstack-ansible contrail-ansible-deployer/playbooks/install_contrailtools.yml

重启

一旦TF完成部署，vRouter内核模块已编译并插入后，可能有必要重启主机以清除vRouter代理容器内遇到的一些问题，包括：

代码语言：javascript复制

contrail-vrouter-agent: controller/src/vnsw/agent/vrouter/ksync/ksync_memory.cc:107: void KSyncMemory::Mmap(bool): Assertion `0' failed.
/entrypoint.sh: line 304: 28142 Aborted (core dumped) $@

我发现在安装后重新启动主机，足以解决这些问题。

部署后的工作

现在我们已经安装了Tungsten Fabric，可以使用 contrail-status命令检查服务的状态：

代码语言：javascript复制

root@aio1:/opt/openstack-ansible/playbooks# contrail-status
Pod Service Original Name State Status
analytics api contrail-analytics-api running Up 21 minutes
analytics collector contrail-analytics-collector running Up 21 minutes
analytics nodemgr contrail-nodemgr running Up 21 minutes
analytics query-engine contrail-analytics-query-engine running Up 21 minutes
config api contrail-controller-config-api running Up 24 minutes
config cassandra contrail-external-cassandra running Up 27 minutes
config device-manager contrail-controller-config-devicemgr running Up 24 minutes
config nodemgr contrail-nodemgr running Up 24 minutes
config rabbitmq contrail-external-rabbitmq running Up 27 minutes
config schema contrail-controller-config-schema running Up 24 minutes
config svc-monitor contrail-controller-config-svcmonitor running Up 24 minutes
config zookeeper contrail-external-zookeeper running Up 27 minutes
control control contrail-controller-control-control running Up 23 minutes
control dns contrail-controller-control-dns running Up 23 minutes
control named contrail-controller-control-named running Up 23 minutes
control nodemgr contrail-nodemgr running Up 23 minutes
database cassandra contrail-external-cassandra running Up 22 minutes
database nodemgr contrail-nodemgr running Up 22 minutes
database zookeeper contrail-external-zookeeper running Up 22 minutes
vrouter agent contrail-vrouter-agent running Up 19 minutes
vrouter nodemgr contrail-nodemgr running Up 19 minutes
webui job contrail-controller-webui-job running Up 23 minutes
webui web contrail-controller-webui-web running Up 23 minutes
vrouter kernel module is PRESENT
== Contrail control ==
control: active
nodemgr: active
named: active
dns: active
== Contrail database ==
nodemgr: active
zookeeper: active
cassandra: active
== Contrail analytics ==
nodemgr: active
api: initializing (UvePartitions:UVE-Aggregation[Partitions:0] connection down)
collector: initializing (KafkaPub:172.29.236.100:9092 connection down)
query-engine: active
== Contrail webui ==
web: active
job: active
== Contrail vrouter ==
nodemgr: active
agent: active
== Contrail config ==
api: active
zookeeper: active
svc-monitor: active
nodemgr: active
device-manager: active
cassandra: active
rabbitmq: active
schema: active

如果看到nodemgr 服务处于Initializing 状态（NTP状态未同步），请尝试重启主机上的NTP。你可能需要定义多个NTP服务器。我尚未解决Analytics的问题，希望能尽快解决。

一些问题

我想说的是，到这里一切都应该可以正常进行了，但事实并非如此！使用OpenStack或Neutron客户端可能无法正常工作，因为neutron-server 服务可能因以下错误而运行失败：

代码语言：javascript复制

Unrecoverable error: please check log for details.: ExtensionsNotFound: Extensions not found: ['route-table']. Related to WARNING neutron.api.extensions [-] Extension file vpcroutetable.py wasn't loaded due to cannot import name attributes.

在Rocky中，一个TF插件所依赖而被OpenStack弃用的模块被删除了。不要担心，它只是移动到了另一个模块/文件中。

在 neutron_server容器中，我修改了/openstack/venvs/neutron-18.0.0.0b3/lib/python2.7/site-packages/neutron_plugin_contrail/extensions/vpcroutetable.py文件，如下所示：

代码语言：javascript复制

-from neutron.api.v2 import attributes as attr
 from neutron_lib.api import attributes as attr

需要重新启动neutron-server服务：

代码语言：javascript复制

# systemctl restart neutron-server

测试验证

Tungsten Fabric已配置为使用Keystone身份验证，要验证这一点，可以使用外部VIP地址和端口8143在浏览器中打开TF的UI：

输入用户名admin，以及在openrc 文件中定义的密码。域的地方填default。 如果身份验证成功，则登录面板应如下显示：

回到OpenStack，我们可以执行一个openstack network list 的命令来查看网络：

代码语言：javascript复制

root@aio1-utility-container-ee37a935:~# openstack network list
 -------------------------------------- ------------------------- --------- 
| ID | Name | Subnets |
 -------------------------------------- ------------------------- --------- 
| 723e67c1-8ccd-43ba-a6f4-8b2399c1b8d2 | __link_local__ | |
| 5a2947ce-0030-4d2a-a06a-76b0d6934102 | ip-fabric | |
| 5f4b0153-8146-4e9c-91d4-c60364ece6bc | default-virtual-network | |
 -------------------------------------- ------------------------- --------- 

这些网络都是由TF插件/驱动程序创建的，不应删除。

创建一个测试网络：

代码语言：javascript复制

root@aio1-utility-container-ee37a935:~# openstack network create test_network_green
 --------------------------- -------------------------------------- 
| Field | Value |
 --------------------------- -------------------------------------- 
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| created_at | None |
| description | None |
| dns_domain | None |
| id | d9e0507f-5ef4-4b62-bf69-176340095053 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| mtu | None |
| name | test_network_green |
| port_security_enabled | True |
| project_id | e565909917a5463b867c5a7594a7612f |
| provider:network_type | None |
| provider:physical_network | None |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | None |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| updated_at | None |
 --------------------------- -------------------------------------- 

注意 provider属性是未指定的，但这些对我们不再重要。TF插件可能不支持其它的属性。

创建子网：

代码语言：javascript复制

root@aio1-utility-container-ee37a935:~# openstack subnet create --subnet-range 172.23.0.0/24 --network test_network_green test_subnet_green
 ------------------- -------------------------------------- 
| Field | Value |
 ------------------- -------------------------------------- 
| allocation_pools | 172.23.0.2-172.23.0.254 |
| cidr | 172.23.0.0/24 |
| created_at | None |
| description | None |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 172.23.0.1 |
| host_routes | |
| id | cc2d2f56-5c87-49fb-afd5-14e32feccd6a |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | test_subnet_green |
| network_id | d9e0507f-5ef4-4b62-bf69-176340095053 |
| project_id | e565909917a5463b867c5a7594a7612f |
| revision_number | None |
| segment_id | None |
| service_types | None |
| subnetpool_id | None |
| tags | |
| updated_at | None |
 ------------------- -------------------------------------- 

IPv6应该是支持的，但是在尝试创建IPv6子网时遇到了问题。这里我们的网络已准备好用于VM。为了达到良好的效果，我创建了一个安全组，该安全组可以应用于允许SSH的实例：

代码语言：javascript复制

root@aio1-utility-container-ee37a935:~# openstack security group create allow_ssh
 ----------------- -------------------------------------- 
| Field | Value |
 ----------------- -------------------------------------- 
| created_at | None |
| description | allow_ssh |
| id | 39a9e241-27c3-452a-b37a-80b6dcbbf783 |
| name | allow_ssh |
| project_id | e565909917a5463b867c5a7594a7612f |
| revision_number | None |
| rules | |
| tags | [] |
| updated_at | None |
 ----------------- -------------------------------------- 
root@aio1-utility-container-ee37a935:~# openstack security group rule create --dst-port 22 allow_ssh
 ------------------- -------------------------------------- 
| Field | Value |
 ------------------- -------------------------------------- 
| created_at | None |
| description | None |
| direction | ingress |
| ether_type | IPv4 |
| id | b8393e4d-1d9d-47e9-877e-86374f38dca1 |
| name | None |
| port_range_max | 22 |
| port_range_min | 22 |
| project_id | e565909917a5463b867c5a7594a7612f |
| protocol | tcp |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | None |
| security_group_id | 39a9e241-27c3-452a-b37a-80b6dcbbf783 |
| updated_at | None |
 ------------------- -------------------------------------- 

随后，我使用tiny flavor和CirrOS镜像启动了实例：

代码语言：javascript复制

root@aio1-utility-container-ee37a935:~# openstack server create --image cirros --flavor test_flavor --nic net-id=test_network_green --security-group allow_ssh test1
 ------------------------------------- ---------------------------------------------------- 
| Field | Value |
 ------------------------------------- ---------------------------------------------------- 
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | a8tghwSoTWZP |
| config_drive | |
| created | 2018-06-18T14:34:49Z |
| flavor | test_flavor (5c0600b7-f9fe-46f3-8af5-f8390ee5c6f3) |
| hostId | |
| id | b14d1861-8855-4d17-a2d3-87eb67a3d81c |
| image | cirros (4006fd58-cdc5-4bd8-bc25-ef73be1cd429) |
| key_name | None |
| name | test1 |
| progress | 0 |
| project_id | e565909917a5463b867c5a7594a7612f |
| properties | |
| security_groups | name='39a9e241-27c3-452a-b37a-80b6dcbbf783' |
| status | BUILD |
| updated | 2018-06-18T14:34:49Z |
| user_id | f6aac1aa53294659998aa71838133a1d |
| volumes_attached | |
 ------------------------------------- ---------------------------------------------------- 
root@aio1-utility-container-ee37a935:~# openstack server list
 -------------------------------------- ------- -------- ------------------------------- -------- ------------- 
| ID | Name | Status | Networks | Image | Flavor |
 -------------------------------------- ------- -------- ------------------------------- -------- ------------- 
| b14d1861-8855-4d17-a2d3-87eb67a3d81c | test1 | ACTIVE | test_network_green=172.23.0.3 | cirros | test_flavor |
 -------------------------------------- ------- -------- ------------------------------- -------- ------------- 

现在，我可以连接到实例的控制台，并尝试出站连接：

在Tungsten Fabric UI中，我能够在网络上启用snat ，以允许vRouter对来自VM的出站连接进行snat：

快速测试显示ping正常工作：

到VM的入站连接也是可行的，但需要Tungsten Fabric进行一些额外的工作才能通告VM地址。在我的实验室中有一个Cisco ASA 1001，已配置为与TF控制器建立对等关系，但我们下一次再展示它是如何配置的吧。

总结

对于学习了解Tungsten Fabric的运行方式，以及围绕如何在基于OpenStack-Ansible的云中部署构建最佳实践，还有很多工作要做。用于安装过程的某些组件，被大量包装在Docker容器中，并且必须先提取才能在LXC容器和/或主机中进行部署。这是不可扩展的，但目前来说已经足够了。

最近，我遇到了与opencontrailnightly 版本有关的问题，vRouter丢弃来自VM的出站或响应流量。借助Juniper repo中的GA版本，该问题已经解决了，但并非每个人都可以使用该访问权限。

我遇到的另一个问题是，在往返于VM的ping工作正常（在中间使用ASR）的同时，SSH却连接失败。实际上，任何TCP连接都失败了。在该实例中看到了SYN，并且观察到发送了SYN/ACK。但是，SYN/ACK从未通过vRouter。抓包信息表明，SYN/ACK的校验和无效。当在主机的“physical”接口上禁用通用IP校验和，这种情况下为ens160 ，可以使一切恢复正常。下面这篇文章超级有帮助：

https://kb.juniper.net/InfoCenter/index?page=content&id=KB30500

随着我获得更多的reps，我希望简化流程，并且能有一天将其移到上游以包含在OpenStack-Ansible中。在那之前，祝我们都好运！

---

作者：James Denton 译者：TF编译组

原文链接：https://www.jimmdenton.com/contrail-osa/

sdn

0 人点赞