https://github.com/apache/atlas (github 拉取对应分支代码)
本次部署所使用版本为2.1-rc3
在编译前确定好所使用外置组件的版本,以免出现版本不兼容问题
当前atlas所依赖组件版本于atlas源码根目录pom.xml中查看
测试所使用的elasticsearch版本为7.3.2(本次测试使用elasticsearch替代solr),kafka为2.12-2.3.0,hbase为2.1.0 cdh6.2.1,zookeeper为3.4.6
1)building atlas(构建atlas):
mvn clean -DskipTests install
2)package atlas(打包atlas):
mvn clean -DskipTests package -Pdist
(注:如需打包入内嵌组件可在后缀加上embedded-组件名 ,示例 : mvn clean -DskipTests package -Pdist,embedded-hbase-solr)
完成后会生成如下包(windows端编译)
编译完成后将apache-atlas-${project.version}-bin.tar.gz 上传至部署机器
tar -xzvf apache-atlas-${project.version}-bin.tar.gz 解压至对应目录
部署机器所需前置步骤:
1)hosts中配置相关主机名映射。
2)确认janusGraph对应组件版本是否兼容(hbase以及elasticsearch)。
3)相关主机能否ping通。
一、解压后进入conf目录下atlas-application.properties 编辑对应配置项
1)修改图库索引为elasticsearch
此处图库索引由默认solr修改为elasticsearch
代码语言:javascript复制# Graph Search Index
atlas.graph.index.search.backend=elasticsearch
注释掉solr相关配置项
配置elasticsearch相关项(此处测试ES所使用双节点)
代码语言:javascript复制atlas.graph.index.search.hostname=esnode1,esnode2
atlas.graph.index.search.elasticsearch.client-only=true
2)Hbase相关配置
代码语言:javascript复制atlas.graph.storage.backend=hbase2
atlas.graph.storage.hbase.table=apache_atlas_janus
代码语言:javascript复制#Hbase
#For standalone mode , specify localhost
#for distributed mode, specify zookeeper quorum here
atlas.graph.storage.hostname=node1:2181,node2:2181,node3:2181
atlas.graph.storage.hbase.regions-per-server=1
atlas.graph.storage.lock.wait-time=10000
代码语言:javascript复制######### Entity Audit Configs #########
atlas.audit.hbase.tablename=apache_atlas_entity_audit
<a
href="http://atlas.audit.zookeeper.session.timeout.ms/">atlas.audit.zookeeper.session.timeout.ms</a>=1000
atlas.audit.hbase.zookeeper.quorum=node1:2181,node2:2181,node3:2181
3)kafka相关配置
代码语言:javascript复制atlas.notification.embedded=false
atlas.kafka.data=/home/service/apache-atlas-2.1.0/data/kafka
atlas.kafka.zookeeper.connect=node1:2181,node2:2181,node3:2181
atlas.kafka.bootstrap.servers=node1:9092,node2:9092,node3:9092
<a href="http://atlas.kafka.zookeeper.session.timeout.ms/">atlas.kafka.zookeeper.session.timeout.ms</a>=400
<a href="http://atlas.kafka.zookeeper.connection.timeout.ms/">atlas.kafka.zookeeper.connection.timeout.ms</a>=200
<a href="http://atlas.kafka.zookeeper.sync.time.ms/">atlas.kafka.zookeeper.sync.time.ms</a>=20
<a href="http://atlas.kafka.auto.commit.interval.ms/">atlas.kafka.auto.commit.interval.ms</a>=1000
<a href="http://atlas.kafka.hook.group.id/">atlas.kafka.hook.group.id</a>=atlas
代码语言:javascript复制atlas.kafka.enable.auto.commit=false
atlas.kafka.auto.offset.reset=earliest
<a href="http://atlas.kafka.session.timeout.ms/">atlas.kafka.session.timeout.ms</a>=30000
atlas.kafka.offsets.topic.replication.factor=1
<a href="http://atlas.kafka.poll.timeout.ms/">atlas.kafka.poll.timeout.ms</a>=1000
atlas.notification.create.topics=true
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.notification.log.failed.messages=true
atlas.notification.consumer.retry.interval=500
atlas.notification.hook.retry.interval=1000
4)atlas HA相关配置
代码语言:javascript复制######### High Availability Configuration ########
atlas.server.ha.enabled=true
#### Enabled the configs below as per need if HA is enabled #####
atlas.server.ids=id1,id2
atlas.server.address.id1=hostname1:21000
atlas.server.address.id1=hostname2:21000
atlas.server.ha.zookeeper.connect=node1:2181,node2:2181,node3:2181
<a href="http://atlas.server.ha.zookeeper.retry.sleeptime.ms/">atlas.server.ha.zookeeper.retry.sleeptime.ms</a>=1000
atlas.server.ha.zookeeper.num.retries=3
<a href="http://atlas.server.ha.zookeeper.session.timeout.ms/">atlas.server.ha.zookeeper.session.timeout.ms</a>=20000
## if ACLs need to be set on the created nodes, uncomment these lines and set the values ##
#atlas.server.ha.zookeeper.acl=<scheme>:<id>
#atlas.server.ha.zookeeper.auth=<scheme>:<authinfo>
主机和从机配置项修改为对应hostname
代码语言:javascript复制######### Server Properties #########
atlas.rest.address=<a href="http://10-177-204-71.lineage-atlas.dgtest01:21000/">hostname:21000</a>
# If enabled and set to true, this will run setup steps when the server starts
二、conf/hbase下 hbase-site.xml 配置
填写atlas安装对应路径
代码语言:javascript复制<configuration>
<property>
<name>hbase.rootdir</name>
<value>${url_prefix}${atlas_data}/hbase-root</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>${atlas_data}/hbase-zookeeper-data</value>
</property>
<property>
<name><a href="http://hbase.master.info/">hbase.master.info</a>.port</name>
<value>61510</value>
</property>
<property>
<name><a href="http://hbase.regionserver.info/">hbase.regionserver.info</a>.port</name>
<value>61530</value>
</property>
<property>
<name>hbase.master.port</name>
<value>61500</value>
</property>
<property>
<name>hbase.regionserver.port</name>
<value>61520</value>
</property>
</configuration>
conf 目录下atlas-env.sh
添加hbase conf目录
export HBASE_CONF_DIR=/home/service/apache-atlas-2.1.0/conf/hbase
三、验证HA是否开启成功
配置完成后主机以及从机启动atlas
bin/目录下执行
./atlas_start.py
查看高可用是否成功
./atlas_admin.py -status
主机显示ACTIVE 状态
从机显示PASSIVE 状态
则成功