(内部资料)第十七讲:如何将服务一劳永逸的集成到ambari中,方便新环境部署,无需二次拷贝

2023-02-21 11:03:47 浏览数 (1)

一、前言

如何将服务一劳永逸的集成到 ambari 中,方便新环境部署,无需二次拷贝呢?

通常,我们在自定义服务集成开发中,会将集成服务的代码项目放置到 /var/lib/ambari-server/resources/stacks/HDP/<version>/services/ 目录下。这样子的前提就是 ambari 与 hdp 集群必须提前搭建好。

其实这种方式是适合开发模式的。但是当集成服务开发、测试好之后,封版的话,再用这种方式就不太好了。假如有几十个自定义服务,你都得挨个手动拷贝,麻烦还容易犯错不是。并且万一要部署的环境再一多,显然这种并不是很好的部署模式。

那怎么办呢?

我们将集成服务的源码整合到 ambari-server rpm 包中不就行了?

我们需要将集成服务的源码放置在 ambari-server 模块的 resources 的对应目录下,然后再通过操作得到新的 ambari-server 的 rpm 即可。这里针对 rpm 包的制作提供了两种方式:

  • 方式一:整体编译 ambari 源码(比较复杂,但是正规)
  • 方式二:重新构建 rpm 包

二、方式一:整体编译 ambari 源码

以 ELASTICSEARCH 服务为例,将 ELASTICSEARCH 相关项目代码拷贝到 hdp 对应的 version 中,然后整体编译,得到新的 ambari-server rpm 包。

整体编译命令:

代码语言:javascript复制
mvn -B -X -e install package rpm:rpm -DnewVersion=2.7.3.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -Preplaceurl

详细可参考我的博客:《Ambari2.7.1整体编译 安装使用》 。

三、方式二:重新构建 rpm 包

如果嫌整体编译 ambari 太繁琐,也可以基于已有的 ambari-server rpm 包,重新构建 rpm 包,这样相对简单一些。

1、首先安装 rpmrebuild 工具

链接:https://pan.baidu.com/s/1SkO2SlLmrl4vWqbgrRuzHg 提取码:m1ti

将 tar 包拷贝到 /opt 下面,编译 rpmrebuild ,貌似 rpm 包与 rpmrebuild 版本有关系。

代码语言:javascript复制
cd /opt
tar zxvf rpmrebuild-2.11.tar.gz 
make 
make install
2、安装工具 rpm-build
代码语言:javascript复制
yum install rpm-build
3、安装 ambari-server

如果环境中没有 ambari-server 服务,可以先安装官方提供的 ambari-server 的 rpm 包

代码语言:javascript复制
yum install ambari-server
4、将集成服务拷贝到指定目录
将自定义的集成服务拷贝到
/var/lib/ambari-server/resources/stacks/HDP/<version>/services/
目录下即可。

1)比如我要将 ELASTICSEARCH 集成到 hdp 3.1 里面,所以我需要将 ELASTICSEARCH 服务代码拷贝到 /var/lib/ambari-server/resources/stacks/HDP/3.1/services/ 目录下。

5、rebuild ambari-server 的 rpm 包
代码语言:javascript复制
rpmrebuild -e ambari-server-2.7.3.0-139.x86_64

在执行 rpmrebuild -e 命令的交互过程中,可修改 .spec 文件,增加 ELASTICSEARCH 的相关文件:

代码语言:javascript复制
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/configuration"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/configuration/elastic-config.xml"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/configuration/elastic-env.xml"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/alerts"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/alerts/alert_check_dir.py"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/cfgNodejs.sh"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/changeOsConfToES.sh"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/createUser.sh"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/elastic_common.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/elasticsearch_client.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/es_metrics.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/head.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/master.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/params.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/service_check.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/status_params.py"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates/changeHostName.sh.j2"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates/data_to_metrics.sh.j2"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates/es_metrics.py.j2"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates/install_requests.sh.j2"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/quicklinks"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/quicklinks/quicklinks-es.json"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/themes"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/themes/theme_elasticsearch_version_4.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/metainfo.xml"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/metrics.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/role_command_order.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/widgets.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/alerts.json"

大概需要 3、4 分钟,rpm 包就重新构建好了。

6、检查新构建的 rpm 包内是否有新添加的文件
代码语言:javascript复制
rpm -qpl ambari-server-2.7.3.0-139.x86_64.rpm
7、安装新构建的 rpm 包

安装下新构建的 rpm 包,测试新集成的 ELASTICSEARCH 服务功能是否正常。

1)停止存在的 ambari-server 进程:ambari-server stop

2)卸载 ambari-server 服务:yum remove ambari-server

3) 安装新构建的 rpm 包:yum install /root/ambari-server-2.7.3.0-139.x86_64.rpm

4)重新设置 ambari-server 配置:ambari-server setup

5)启动 ambari-server 进程

6)重装 ELASTICSEARCH 服务,测试功能。

本文对应的视频已上传至 bilibili 平台,可点击原文链接或拷贝以下链接观看:

https://www.bilibili.com/video/BV1xz4y117K4/

关于对 Ambari 自定义服务集成实战课程的详细介绍,可以拷贝以下链接观看:

https://www.yuque.com/create17/ambari/miyk6c

《Ambari 源码编译及前后端二次开发实战》课程,详情介绍:

https://www.yuque.com/create17/mxswdh/xpoa10

0 人点赞