druid调参

2022-02-22 00:11:10 浏览数 (1)

如下是推荐参数： broker 可以不用很大

代码语言：javascript复制
broker-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms32g
-Xmx32g
-XX:MaxDirectMemorySize=20g
-Djute.maxbuffer=41943040

broker-runtime.properties
------------------------------------------------------------------------------------------------------------
druid.processing.numMergeBuffers 12
druid.processing.numThreads 48
druid.server.http.numThreads 85

common

代码语言：javascript复制
common.runtime.properties
------------------------------------------------------------------------------------------------------------
druid.indexer.logs.directory  hdfs://HDFSxxxx/druid/indexing-logs
druid.storage.storageDirectory hdfs://HDFSxxxxx/druid/segments

coordinator

代码语言：javascript复制
coordinator-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms12g
-Xmx12g
-Djute.maxbuffer=41943040

historical分冷热节点 不同节点可以参考评论中的配置 historical冷节点

代码语言：javascript复制
historical-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms100g
-Xmx100g
-XX:MaxDirectMemorySize=100g
-Djute.maxbuffer=41943040

historical-runtime.properties
------------------------------------------------------------------------------------------------------------
druid.processing.numMergeBuffers 12
druid.processing.numThreads 48
druid.segmentCache.locations [{"path":"/data1/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data2/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data3/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data4/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data5/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data6/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data7/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data8/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data9/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data10/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data11/druid/var/druid/segment-cache","maxSize":10000000000000},{"path":"/data12/druid/var/druid/segment-cache","maxSize":10000000000000}]
druid.server.http.numThreads 85
druid.server.maxSize 120000000000000
druid.server.tier cold

historical热节点

代码语言：javascript复制
historical-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms100g
-Xmx100g
-XX:MaxDirectMemorySize=100g
-Djute.maxbuffer=41943040

historical-runtime.properties
------------------------------------------------------------------------------------------------------------
druid.processing.numMergeBuffers 12
druid.processing.numThreads 48
druid.segmentCache.locations [{"path":"/data/druid/var/druid/segment-cache","maxSize":4000000000000}]
druid.server.http.numThreads 85
druid.server.maxSize 4000000000000
druid.server.priority 100

middleManager

代码语言：javascript复制
middleManager-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms128m
-Xmx128m
-Djute.maxbuffer=41943040

middleManager-runtime.properties
------------------------------------------------------------------------------------------------------------
druid.indexer.task.baseTaskDir /data/druid/var/druid/task
druid.server.http.numThreads 25
druid.worker.capacity 40

overlord

代码语言：javascript复制
overlord-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms12g
-Xmx12g
-Djute.maxbuffer=41943040

router

代码语言：javascript复制
router-jvm.config
------------------------------------------------------------------------------------------------------------
-Xms1g
-Xmx4g
-XX:MaxDirectMemorySize=512m
-Djute.maxbuffer=41943040


router-runtime.properties 
------------------------------------------------------------------------------------------------------------
druid.plaintextPort 8080 （上机器修改）

hadoop 参数和xml

  • druid和hdfs分开 http://yun.isd.com/index.php/package/versions/?product=other&package=druid_hdfs_conf cd /usr/local/service/druid/conf/druid/cluster/_common; rm *site.xml; cp /usr/local/services/druid_hdfs_conf-1.0/conf/hdfs42717/* . cd /usr/local/service/druid/conf/druid/_common; rm *site.xml; cp /usr/local/services/druid_hdfs_conf-1.0/conf/hdfs42717/* .
  • 调整hdfs参数 hadoop-env.sh DNHeapsize 20480 Heapsize 10240 NNHeapsize 30720

ZK

  • 调整堆内存 zkEnv.sh Heapsize 8192m

重启

-先重启zk

  • 重启hdfs
  • 重启druid
node.js 大数据 hadoop xml bash

0 人点赞