大数据ELK Stack（二）：Elasticsearch 集群之docker部署

2022-09-17 15:17:55 浏览数 (1)

一、环境配置

版本

Linux环境：CentOS Linux release 7.9.2009 (Core)

Java：openjdk version 1.8.0_332 (build 25.332-b09, mixed mode)

Elasticsearch版本：7.14.2

配置

节点数量：3

内存：8G

硬盘：30G

CPU个数：1

CPU核心数：2

CPU型号：Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz

二、单节点部署

  • 拉取镜像
代码语言：javascript复制
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.14.2
  • docker启动单节点
代码语言：javascript复制
docker run -p 127.0.0.1:9200:9200 -p 127.0.0.1:9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.14.2
代码语言：javascript复制
[root@localhost ~]# docker ps
CONTAINER ID   IMAGE                                                  COMMAND                  CREATED              STATUS              PORTS                                                NAMES
4b6ca4799d88   docker.elastic.co/elasticsearch/elasticsearch:7.14.2   "/bin/tini -- /usr/l…"   About a minute ago   Up About a minute   127.0.0.1:9200->9200/tcp, 127.0.0.1:9300->9300/tcp   inspiring_dijkstra
代码语言：javascript复制
[root@localhost ~]# curl -XGET 127.0.0.1:9200
{
  "name" : "4b6ca4799d88",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "p-H8BxaOTcqunC3BZ-ZZUg",
  "version" : {
    "number" : "7.14.2",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "6bc13727ce758c0e943c3c21653b3da82f627f75",
    "build_date" : "2021-09-15T10:18:09.722761972Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

三、使用docker-compuse部署多节点集群

  • 创建docker-compose.yml
代码语言：javascript复制
[root@localhost ElasticSearch]# cat docker-compose.yml
version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.2
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
      - log01:/usr/share/elasticsearch/log
    ports:
      - 9200:9200
    networks:
      - elastic
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.2
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
      - log02:/usr/share/elasticsearch/log
    networks:
      - elastic
  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.2
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
      - log03:/usr/share/elasticsearch/log
    networks:
      - elastic
  kibana:
    image: docker.elastic.co/kibana/kibana:7.14.2
    container_name: kibana
    volumes:
      - /opt/ElasticSearch/kibana.yml:/usr/share/kibana/config/kibana.yml
    ports:
      - 5601:5601
    networks:
      - elastic

volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local
  log01:
    driver: local
  log02:
    driver: local
  log03:
    driver: local
  kibana.yml:
    driver: local

networks:
  elastic:
    driver: bridge
  • 运行docker-compose启动集群（后台启动-d）
代码语言：javascript复制
docker-compose up
代码语言：javascript复制
[root@localhost ElasticSearch]# docker-compose up
[ ] Running 4/0
 ⠿ Container es02    Created                                                                                                                                                                             0.0s
 ⠿ Container kibana  Created                                                                                                                                                                             0.0s
 ⠿ Container es03    Created                                                                                                                                                                             0.0s
 ⠿ Container es01    Created                                                                                                                                                                             0.0s
Attaching to es01, es02, es03, kibana
  • 提交_cat/nodes请求以查看节点是否已启动并正在运行
代码语言：javascript复制
[root@localhost ~]# curl -X GET "localhost:9200/_cat/nodes?v=true&pretty"
ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
172.18.0.2           62          88   9    0.33    0.78     0.57 cdfhilmrstw -      es03
172.18.0.5           35          88   9    0.33    0.78     0.57 cdfhilmrstw -      es02
172.18.0.3           41          88   8    0.33    0.78     0.57 cdfhilmrstw *      es01

四、访问kibana

五、在生产环境中在 Docker 中运行 Elasticsearch 时适用以下要求和建议。

  • 配置文件必须是elasticsearch用户可读的：
代码语言：javascript复制
mkdir es
chmod g rwx es
chgrp 0 es
  • 内核设置必须设置为vm.max_map_count至少262144用于生产用途
代码语言：javascript复制
vim /etc/sysctl.conf
vm.max_map_count=262144
  • 配置文件描述符
代码语言：javascript复制
ulimit -n 65535  #临时修改
vim /etc/security/limits.conf #永久修改
es         soft    nproc     65535
es         hard    nproc     65535
  • 禁用sawpping
代码语言：javascript复制
swapoff -a #临时禁用所有的swap文件
vim /etc/fstab #注释掉所有的swap相关的行，永久禁用
  • 配置线程数
代码语言：javascript复制
ulimit -u 4096 #临时修改
vim /etc/security/limits.conf #永久修改
  • 手动设置 JVM 堆大小 要在生产环境中手动设置堆大小，在其中绑定挂载一个JVM 选项文件/usr/share/elasticsearch/config/jvm.options
ElasticsearchService

0 人点赞