如何使用curl命令调用CM的API动态配置Yarn资源池

2018-08-17 17:27:50 浏览数 (1)

温馨提示：如果使用电脑查看图片不清晰，可以使用手机打开文章单击文中的图片放大查看高清原图。

Fayson的github： https://github.com/fayson/cdhproject

提示：代码块部分可以左右滑动查看噢

1.文档编写目的

在使用CDH集群大数据平台过程中，用户会有需求在自己的统一管理平台上通过API接口能够动态的设置Yarn资源池，Cloudera Manager提供了丰富的API接口对CDH集群的各个服务进行配置等操作，本篇文章Fayson主要介绍如何使用curl命令调用CM的API接口动态的配置Yarn资源池并使其生效。

  • 本次Fayson的测试环境

1.CM和CDH版本为5.15

2.获取Cloudera Manager的API接口

1.登录Cloudera Manager，点击菜单“支持”->“API文档”

2.进入API文档界面，可以看到API的版本为v19

3.本篇文章要用到的API接口说明

代码语言：javascript复制
http://<cm_server>:7180/clusters
http://<cm_server>:7180/api/v19/clusters/{clusterName}/services/{serviceName}/config
http://<cm_server>:7180/api/v19/clusters/{clusterName}/commands/poolsRefresh

（可左右滑动）

共使用到3个接口：

第一个接口用于获取集群的信息，如集群的名称，以供2、3接口使用

第二个接口用于设置Yarn的资源池，接口中的{serviceName}修改为yarn

第三个接口用于刷新Yarn的资源池

3.设置资源池

1.在设置Yarn资源池前配置如下：

2.在命令行使用curl GET的方式通过API接口获取集群信息

代码语言：javascript复制
[root@cdh01 ~]# curl --insecure -X GET -u admin:admin "http://cdh01.fayson.com:7180/api/v19/clusters"

（可左右滑动）

3.在命令行使用curl PUT的方式向指定的API接口提交JSON数据

JSON数据如下：

代码语言：javascript复制
{
   "items": [{
      "name": "yarn_fs_scheduled_allocations",
      "value": "{"defaultFairSharePreemptionThreshold":null,"defaultFairSharePreemptionTimeout":null,"defaultMinSharePreemptionTimeout":null,"defaultQueueSchedulingPolicy":"fair","queueMaxAMShareDefault":null,"queueMaxAppsDefault":null,"queuePlacementRules":[{"create":true,"name":"specified","queue":null,"rules":null},{"create":null,"name":"nestedUserQueue","queue":null,"rules":[{"create":true,"name":"default","queue":"users","rules":null}]},{"create":null,"name":"default","queue":null,"rules":null}],"queues":[{"aclAdministerApps":"*","aclSubmitApps":"*","allowPreemptionFrom":null,"fairSharePreemptionThreshold":null,"fairSharePreemptionTimeout":null,"minSharePreemptionTimeout":null,"name":"root","queues":[{"aclAdministerApps":null,"aclSubmitApps":null,"allowPreemptionFrom":null,"fairSharePreemptionThreshold":null,"fairSharePreemptionTimeout":null,"minSharePreemptionTimeout":null,"name":"default","queues":[],"schedulablePropertiesList":[{"impalaDefaultQueryMemLimit":null,"impalaDefaultQueryOptions":null,"impalaMaxMemory":null,"impalaMaxQueuedQueries":null,"impalaMaxRunningQueries":null,"impalaQueueTimeout":null,"maxAMShare":null,"maxChildResources":null,"maxResources":null,"maxRunningApps":null,"minResources":null,"scheduleName":"default","weight":1.0}],"schedulingPolicy":"drf","type":null},{"aclAdministerApps":null,"aclSubmitApps":null,"allowPreemptionFrom":null,"fairSharePreemptionThreshold":null,"fairSharePreemptionTimeout":null,"minSharePreemptionTimeout":null,"name":"users","queues":[],"schedulablePropertiesList":[{"impalaDefaultQueryMemLimit":null,"impalaDefaultQueryOptions":null,"impalaMaxMemory":null,"impalaMaxQueuedQueries":null,"impalaMaxRunningQueries":null,"impalaQueueTimeout":null,"maxAMShare":null,"maxChildResources":null,"maxResources":null,"maxRunningApps":null,"minResources":null,"scheduleName":"default","weight":2.0}],"schedulingPolicy":"drf","type":"parent"}],"schedulablePropertiesList":[{"impalaDefaultQueryMemLimit":null,"impalaDefaultQueryOptions":null,"impalaMaxMemory":null,"impalaMaxQueuedQueries":null,"impalaMaxRunningQueries":null,"impalaQueueTimeout":null,"maxAMShare":null,"maxChildResources":null,"maxResources":null,"maxRunningApps":null,"minResources":null,"scheduleName":"default","weight":1.0}],"schedulingPolicy":"drf","type":null}],"userMaxAppsDefault":null,"users":[]}"
   }]
}

（可左右滑动）

在命令行执行如下命令设置Yarn资源池：

代码语言：javascript复制
curl --insecure -X PUT -H 'Content-Type:application/json' -u admin:admin -d '定义好的JSON数据' "http://cdh01.fayson.com:7180/api/v10/clusters/cluster/services/yarn/config"

（可左右滑动）

命令执行成功，查看Yarn资源配置已更新

4.完成了上述操作后，资源池的设置还未生效，需要调用刷新资源池接口使其生效

代码语言：javascript复制
[root@cdh01 ~]# curl --insecure -X POST -u admin:admin "http://cdh01.fayson.com:7180/api/v19/clusters/cluster/commands/poolsRefresh"

（可左右滑动）

4.总结

1.通过API接口动态的配置Yarn的资源池，首先要获取Yarn服务所在集群名称，通过接口进行动态设置，设置完成后需要调用刷新的API接口使其生效。

2.在设置Yarn资源池时调用的API接口为PUT方式提交，如果指定其他方式会导致设置失败。

3.设置资源池的JSON字符串，需要注意value跟的数据为转义的字符串，必须带“”,否则会设置失败。

提示：代码块部分可以左右滑动查看噢

为天地立心，为生民立命，为往圣继绝学，为万世开太平。 温馨提示：如果使用电脑查看图片不清晰，可以使用手机打开文章单击文中的图片放大查看高清原图。

原创文章，欢迎转载，转载请注明：转载自微信公众号Hadoop实操

api yarn github https

0 人点赞