Elasticsearch 7.10.1集群压测报告（8核32G*3，Intel）

2022-05-16 15:57:36 浏览数 (1)

说明

本文描述问题及解决方法同样适用于 腾讯云 Elasticsearch Service（ES）。

另外使用到：腾讯云 云服务器（Cloud Virtual Machine，CVM）

本文延续上一篇 Elasticsearch压测工具esrally部署指南

环境配置

Esrally客户端环境

  • 版本

Linux环境：Centos 7.9

Python：3.8.7

Pip：pip 20.2.3 from pip (python 3.8)

Java：openjdk version 1.8.0_302 (build 1.8.0_302-b08)

Git：2.7.5

Esrally：2.3.0

  • 配置

内存：32G

硬盘：SSD云硬盘 100GB

CPU个数：1

CPU核心数：16

Elasticsearch服务端环境

  • 版本

Linux环境：Centos 7.2

Java：openjdk version 11.0.9.1-ga (build 11.0.9.1-ga 1, mixed mode)

Elasticsearch版本：7.10.1（腾讯云 Elasticsearch Service 白金版）

  • 配置

节点数量：3

内存：32G

硬盘：SSD云硬盘 1TB

CPU个数：1

CPU核心数：8

CPU型号：Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50GHz

背景

在大数据时代的今天，业务量越来越大，每天动辄都会产生上百GB、上TB的数据，所以拥有一个性能强劲的Elasticsearch集群就显得尤为重要。我们需要模拟大量网络日志、用户行为日志的读写动作，衡量各性能的指标，找出集群瓶颈所在，以确认我们需要怎样的硬件配置以及业务优化，才能满足现有的业务量，这就是我们在业务上线前所必要做的。

压测

esrally 相关术语及参数

Rally 是汽车拉力赛的意思，所以关于它里面术语也是跟汽车的拉力赛有关。

  • track: 即赛道的意思，这里指压测用到的样本数据和压测策略，使用 esrally list tracks 列出。rally 自带的 track 可在 https://github.com/elastic/rally-tracks 中查看，每个 track 的文件名中都存在 README.md 对压测的数据类型和参数做了详细的说明。如果没有指定 track， 则默认使用 geonames track 进行测试；
  • target-hosts：即远程elasticsearch的ip和端口，以ip:port的形式指定；
  • pipeline: 指一个压测流程，可以通过 esrally list pipeline 查看，其中有一个 benchmark-only 的流程，就是将 es 的管理交给用户来操作，rally 只用来做压测，如果你想针对已有的 es 进行压测，则使用该模式；
  • track-params：对默认的压测参数进行覆盖；
  • user-tag：本次压测的 tag 标记；
  • client-options：指定一些客户端连接选项，比如用户名和密码。

压测指令

代码语言：javascript复制
esrally race 
  --track=geonames 
  --target-hosts=10.0.10.4:9200 
  --pipeline=benchmark-only 
  --track-params="number_of_shards:3, number_of_replicas:1" 
  --user-tag="version:Intel_8C32G_1T*3" 
  --client-options="basic_auth_user:'elastic', basic_auth_password:'your_password'"

压测报告

压测指标

压测任务

压测结果

单位

Cumulative indexing time of primary shards

17.8971

min

Min cumulative indexing time across primary shards

0

min

Median cumulative indexing time across primary shards

0.0014

min

Max cumulative indexing time across primary shards

6.10697

min

Cumulative indexing throttle time of primary shards

0

min

Min cumulative indexing throttle time across primary shards

0

min

Median cumulative indexing throttle time across primary shards

0

min

Max cumulative indexing throttle time across primary shards

0

min

Cumulative merge time of primary shards

4.5751

min

Cumulative merge count of primary shards

122

Min cumulative merge time across primary shards

0

min

Median cumulative merge time across primary shards

0.001492

min

Max cumulative merge time across primary shards

1.52028

min

Cumulative merge throttle time of primary shards

0.792833

min

Min cumulative merge throttle time across primary shards

0

min

Median cumulative merge throttle time across primary shards

0

min

Max cumulative merge throttle time across primary shards

0.3203

min

Cumulative refresh time of primary shards

1.34323

min

Cumulative refresh count of primary shards

1024

Min cumulative refresh time across primary shards

0

min

Median cumulative refresh time across primary shards

0.0092

min

Max cumulative refresh time across primary shards

0.44665

min

Cumulative flush time of primary shards

0.304667

min

Cumulative flush count of primary shards

21

Min cumulative flush time across primary shards

0

min

Median cumulative flush time across primary shards

0.000325

min

Max cumulative flush time across primary shards

0.131633

min

Total Young Gen GC time

12.838

s

Total Young Gen GC count

1058

Total Old Gen GC time

0

s

Total Old Gen GC count

0

Store size

6.05677

GB

Translog size

0.014312

GB

Heap used for segments

0.786037

MB

Heap used for doc values

0.15276

MB

Heap used for terms

0.535995

MB

Heap used for norms

0.049255

MB

Heap used for points

0

MB

Heap used for stored fields

0.048027

MB

Segment count

99

Min Throughput

index-append

66473.4

docs/s

Mean Throughput

index-append

67327.3

docs/s

Median Throughput

index-append

67308.6

docs/s

Max Throughput

index-append

68484.4

docs/s

50th percentile latency

index-append

356.13

ms

90th percentile latency

index-append

1044.3

ms

99th percentile latency

index-append

2036.47

ms

100th percentile latency

index-append

2761.67

ms

50th percentile service time

index-append

356.13

ms

90th percentile service time

index-append

1044.3

ms

99th percentile service time

index-append

2036.47

ms

100th percentile service time

index-append

2761.67

ms

error rate

index-append

0

%

Min Throughput

index-stats

89.81

ops/s

Mean Throughput

index-stats

89.89

ops/s

Median Throughput

index-stats

89.9

ops/s

Max Throughput

index-stats

89.93

ops/s

50th percentile latency

index-stats

3.76232

ms

90th percentile latency

index-stats

4.31119

ms

99th percentile latency

index-stats

25.0359

ms

99.9th percentile latency

index-stats

87.6608

ms

100th percentile latency

index-stats

93.8337

ms

50th percentile service time

index-stats

3.01051

ms

90th percentile service time

index-stats

3.38244

ms

99th percentile service time

index-stats

4.01293

ms

99.9th percentile service time

index-stats

9.50633

ms

100th percentile service time

index-stats

93.4013

ms

error rate

index-stats

0

%

Min Throughput

node-stats

89.91

ops/s

Mean Throughput

node-stats

89.96

ops/s

Median Throughput

node-stats

89.97

ops/s

Max Throughput

node-stats

89.99

ops/s

50th percentile latency

node-stats

4.0404

ms

90th percentile latency

node-stats

4.66332

ms

99th percentile latency

node-stats

5.75505

ms

99.9th percentile latency

node-stats

7.72751

ms

100th percentile latency

node-stats

13.6095

ms

50th percentile service time

node-stats

3.25454

ms

90th percentile service time

node-stats

3.85935

ms

99th percentile service time

node-stats

5.06827

ms

99.9th percentile service time

node-stats

6.71758

ms

100th percentile service time

node-stats

13.1472

ms

error rate

node-stats

0

%

Min Throughput

default

50.01

ops/s

Mean Throughput

default

50.01

ops/s

Median Throughput

default

50.01

ops/s

Max Throughput

default

50.02

ops/s

50th percentile latency

default

4.35737

ms

90th percentile latency

default

4.88184

ms

99th percentile latency

default

5.54453

ms

99.9th percentile latency

default

7.18094

ms

100th percentile latency

default

9.56067

ms

50th percentile service time

default

3.6539

ms

90th percentile service time

default

4.03433

ms

99th percentile service time

default

4.54517

ms

99.9th percentile service time

default

6.37158

ms

100th percentile service time

default

8.47822

ms

error rate

default

0

%

Min Throughput

term

99.96

ops/s

Mean Throughput

term

99.97

ops/s

Median Throughput

term

99.98

ops/s

Max Throughput

term

99.98

ops/s

50th percentile latency

term

3.43838

ms

90th percentile latency

term

3.97576

ms

99th percentile latency

term

4.53968

ms

99.9th percentile latency

term

12.8783

ms

100th percentile latency

term

19.7935

ms

50th percentile service time

term

2.76449

ms

90th percentile service time

term

3.00772

ms

99th percentile service time

term

3.53801

ms

99.9th percentile service time

term

3.92616

ms

100th percentile service time

term

19.4717

ms

error rate

term

0

%

Min Throughput

phrase

109.71

ops/s

Mean Throughput

phrase

109.83

ops/s

Median Throughput

phrase

109.85

ops/s

Max Throughput

phrase

109.89

ops/s

50th percentile latency

phrase

3.65152

ms

90th percentile latency

phrase

4.11519

ms

99th percentile latency

phrase

4.68737

ms

99.9th percentile latency

phrase

15.813

ms

100th percentile latency

phrase

21.5058

ms

50th percentile service time

phrase

2.94523

ms

90th percentile service time

phrase

3.15831

ms

99th percentile service time

phrase

3.52894

ms

99.9th percentile service time

phrase

5.46572

ms

100th percentile service time

phrase

20.4534

ms

error rate

phrase

0

%

Min Throughput

country_agg_uncached

2.99

ops/s

Mean Throughput

country_agg_uncached

2.99

ops/s

Median Throughput

country_agg_uncached

2.99

ops/s

Max Throughput

country_agg_uncached

3

ops/s

50th percentile latency

country_agg_uncached

278.419

ms

90th percentile latency

country_agg_uncached

338.071

ms

99th percentile latency

country_agg_uncached

355.738

ms

100th percentile latency

country_agg_uncached

368.443

ms

50th percentile service time

country_agg_uncached

276.9

ms

90th percentile service time

country_agg_uncached

337.805

ms

99th percentile service time

country_agg_uncached

355.353

ms

100th percentile service time

country_agg_uncached

366.932

ms

error rate

country_agg_uncached

0

%

Min Throughput

country_agg_cached

97.54

ops/s

Mean Throughput

country_agg_cached

98.19

ops/s

Median Throughput

country_agg_cached

98.26

ops/s

Max Throughput

country_agg_cached

98.65

ops/s

50th percentile latency

country_agg_cached

2.5026

ms

90th percentile latency

country_agg_cached

2.9092

ms

99th percentile latency

country_agg_cached

3.9038

ms

99.9th percentile latency

country_agg_cached

8.04707

ms

100th percentile latency

country_agg_cached

8.11013

ms

50th percentile service time

country_agg_cached

1.7528

ms

90th percentile service time

country_agg_cached

1.93257

ms

99th percentile service time

country_agg_cached

2.15279

ms

99.9th percentile service time

country_agg_cached

7.4365

ms

100th percentile service time

country_agg_cached

7.84956

ms

error rate

country_agg_cached

0

%

Min Throughput

scroll

20.03

pages/s

Mean Throughput

scroll

20.03

pages/s

Median Throughput

scroll

20.03

pages/s

Max Throughput

scroll

20.04

pages/s

50th percentile latency

scroll

640.116

ms

90th percentile latency

scroll

651.903

ms

99th percentile latency

scroll

666.913

ms

100th percentile latency

scroll

668.94

ms

50th percentile service time

scroll

638.165

ms

90th percentile service time

scroll

650.977

ms

99th percentile service time

scroll

665.838

ms

100th percentile service time

scroll

667.555

ms

error rate

scroll

0

%

Min Throughput

expression

1.5

ops/s

Mean Throughput

expression

1.5

ops/s

Median Throughput

expression

1.5

ops/s

Max Throughput

expression

1.5

ops/s

50th percentile latency

expression

502.979

ms

90th percentile latency

expression

568.035

ms

99th percentile latency

expression

712.112

ms

100th percentile latency

expression

749.714

ms

50th percentile service time

expression

501.539

ms

90th percentile service time

expression

564.89

ms

99th percentile service time

expression

711.292

ms

100th percentile service time

expression

748.878

ms

error rate

expression

0

%

Min Throughput

painless_static

1.4

ops/s

Mean Throughput

painless_static

1.4

ops/s

Median Throughput

painless_static

1.4

ops/s

Max Throughput

painless_static

1.4

ops/s

50th percentile latency

painless_static

654.669

ms

90th percentile latency

painless_static

666.219

ms

99th percentile latency

painless_static

671.554

ms

100th percentile latency

painless_static

675.303

ms

50th percentile service time

painless_static

653.577

ms

90th percentile service time

painless_static

665.386

ms

99th percentile service time

painless_static

671.186

ms

100th percentile service time

painless_static

674.545

ms

error rate

painless_static

0

%

Min Throughput

painless_dynamic

1.4

ops/s

Mean Throughput

painless_dynamic

1.4

ops/s

Median Throughput

painless_dynamic

1.4

ops/s

Max Throughput

painless_dynamic

1.4

ops/s

50th percentile latency

painless_dynamic

663.227

ms

90th percentile latency

painless_dynamic

700.887

ms

99th percentile latency

painless_dynamic

741.149

ms

100th percentile latency

painless_dynamic

749.156

ms

50th percentile service time

painless_dynamic

661.502

ms

90th percentile service time

painless_dynamic

698.051

ms

99th percentile service time

painless_dynamic

740.17

ms

100th percentile service time

painless_dynamic

748.57

ms

error rate

painless_dynamic

0

%

Min Throughput

decay_geo_gauss_function_score

1

ops/s

Mean Throughput

decay_geo_gauss_function_score

1

ops/s

Median Throughput

decay_geo_gauss_function_score

1

ops/s

Max Throughput

decay_geo_gauss_function_score

1

ops/s

50th percentile latency

decay_geo_gauss_function_score

637.423

ms

90th percentile latency

decay_geo_gauss_function_score

677.084

ms

99th percentile latency

decay_geo_gauss_function_score

689.839

ms

100th percentile latency

decay_geo_gauss_function_score

692.158

ms

50th percentile service time

decay_geo_gauss_function_score

636.484

ms

90th percentile service time

decay_geo_gauss_function_score

675.874

ms

99th percentile service time

decay_geo_gauss_function_score

688.657

ms

100th percentile service time

decay_geo_gauss_function_score

691.315

ms

error rate

decay_geo_gauss_function_score

0

%

Min Throughput

decay_geo_gauss_script_score

1

ops/s

Mean Throughput

decay_geo_gauss_script_score

1

ops/s

Median Throughput

decay_geo_gauss_script_score

1

ops/s

Max Throughput

decay_geo_gauss_script_score

1

ops/s

50th percentile latency

decay_geo_gauss_script_score

642.414

ms

90th percentile latency

decay_geo_gauss_script_score

669.832

ms

99th percentile latency

decay_geo_gauss_script_score

692.145

ms

100th percentile latency

decay_geo_gauss_script_score

694.334

ms

50th percentile service time

decay_geo_gauss_script_score

641.567

ms

90th percentile service time

decay_geo_gauss_script_score

668.511

ms

99th percentile service time

decay_geo_gauss_script_score

690.854

ms

100th percentile service time

decay_geo_gauss_script_score

693.47

ms

error rate

decay_geo_gauss_script_score

0

%

Min Throughput

field_value_function_score

1.5

ops/s

Mean Throughput

field_value_function_score

1.5

ops/s

Median Throughput

field_value_function_score

1.5

ops/s

Max Throughput

field_value_function_score

1.5

ops/s

50th percentile latency

field_value_function_score

254.779

ms

90th percentile latency

field_value_function_score

301.639

ms

99th percentile latency

field_value_function_score

344.603

ms

100th percentile latency

field_value_function_score

354.755

ms

50th percentile service time

field_value_function_score

252.933

ms

90th percentile service time

field_value_function_score

299.199

ms

99th percentile service time

field_value_function_score

342.207

ms

100th percentile service time

field_value_function_score

352.667

ms

error rate

field_value_function_score

0

%

Min Throughput

field_value_script_score

1.5

ops/s

Mean Throughput

field_value_script_score

1.5

ops/s

Median Throughput

field_value_script_score

1.5

ops/s

Max Throughput

field_value_script_score

1.5

ops/s

50th percentile latency

field_value_script_score

324.439

ms

90th percentile latency

field_value_script_score

383.399

ms

99th percentile latency

field_value_script_score

396.528

ms

100th percentile latency

field_value_script_score

429.46

ms

50th percentile service time

field_value_script_score

322.952

ms

90th percentile service time

field_value_script_score

382.208

ms

99th percentile service time

field_value_script_score

395.474

ms

100th percentile service time

field_value_script_score

428.324

ms

error rate

field_value_script_score

0

%

Min Throughput

large_terms

1

ops/s

Mean Throughput

large_terms

1.01

ops/s

Median Throughput

large_terms

1.01

ops/s

Max Throughput

large_terms

1.01

ops/s

50th percentile latency

large_terms

21242.9

ms

90th percentile latency

large_terms

24030.6

ms

99th percentile latency

large_terms

24543.6

ms

100th percentile latency

large_terms

24653.5

ms

50th percentile service time

large_terms

1018.05

ms

90th percentile service time

large_terms

1033.78

ms

99th percentile service time

large_terms

1055.77

ms

100th percentile service time

large_terms

1085.81

ms

error rate

large_terms

0

%

Min Throughput

large_filtered_terms

1.04

ops/s

Mean Throughput

large_filtered_terms

1.04

ops/s

Median Throughput

large_filtered_terms

1.04

ops/s

Max Throughput

large_filtered_terms

1.04

ops/s

50th percentile latency

large_filtered_terms

13956.2

ms

90th percentile latency

large_filtered_terms

15970.5

ms

99th percentile latency

large_filtered_terms

16585.8

ms

100th percentile latency

large_filtered_terms

16594.7

ms

50th percentile service time

large_filtered_terms

999.665

ms

90th percentile service time

large_filtered_terms

1014.51

ms

99th percentile service time

large_filtered_terms

1030.27

ms

100th percentile service time

large_filtered_terms

1052.65

ms

error rate

large_filtered_terms

0

%

Min Throughput

large_prohibited_terms

0.96

ops/s

Mean Throughput

large_prohibited_terms

0.96

ops/s

Median Throughput

large_prohibited_terms

0.96

ops/s

Max Throughput

large_prohibited_terms

0.96

ops/s

50th percentile latency

large_prohibited_terms

34937.3

ms

90th percentile latency

large_prohibited_terms

40304.5

ms

99th percentile latency

large_prohibited_terms

41289.4

ms

100th percentile latency

large_prohibited_terms

41383.7

ms

50th percentile service time

large_prohibited_terms

1023.82

ms

90th percentile service time

large_prohibited_terms

1125.19

ms

99th percentile service time

large_prohibited_terms

1233.24

ms

100th percentile service time

large_prohibited_terms

1373.52

ms

error rate

large_prohibited_terms

0

%

Min Throughput

desc_sort_population

1.5

ops/s

Mean Throughput

desc_sort_population

1.5

ops/s

Median Throughput

desc_sort_population

1.5

ops/s

Max Throughput

desc_sort_population

1.51

ops/s

50th percentile latency

desc_sort_population

113.633

ms

90th percentile latency

desc_sort_population

188.21

ms

99th percentile latency

desc_sort_population

205.838

ms

100th percentile latency

desc_sort_population

206.634

ms

50th percentile service time

desc_sort_population

112.221

ms

90th percentile service time

desc_sort_population

186.685

ms

99th percentile service time

desc_sort_population

204.112

ms

100th percentile service time

desc_sort_population

205.142

ms

error rate

desc_sort_population

0

%

Min Throughput

asc_sort_population

1.5

ops/s

Mean Throughput

asc_sort_population

1.51

ops/s

Median Throughput

asc_sort_population

1.5

ops/s

Max Throughput

asc_sort_population

1.51

ops/s

50th percentile latency

asc_sort_population

110.387

ms

90th percentile latency

asc_sort_population

187.696

ms

99th percentile latency

asc_sort_population

193.171

ms

100th percentile latency

asc_sort_population

195.723

ms

50th percentile service time

asc_sort_population

109.176

ms

90th percentile service time

asc_sort_population

186.048

ms

99th percentile service time

asc_sort_population

192.03

ms

100th percentile service time

asc_sort_population

194.855

ms

error rate

asc_sort_population

0

%

Min Throughput

asc_sort_with_after_population

1.5

ops/s

Mean Throughput

asc_sort_with_after_population

1.5

ops/s

Median Throughput

asc_sort_with_after_population

1.5

ops/s

Max Throughput

asc_sort_with_after_population

1.51

ops/s

50th percentile latency

asc_sort_with_after_population

159.499

ms

90th percentile latency

asc_sort_with_after_population

229.463

ms

99th percentile latency

asc_sort_with_after_population

250.332

ms

100th percentile latency

asc_sort_with_after_population

286.187

ms

50th percentile service time

asc_sort_with_after_population

158.401

ms

90th percentile service time

asc_sort_with_after_population

228.038

ms

99th percentile service time

asc_sort_with_after_population

249.041

ms

100th percentile service time

asc_sort_with_after_population

284.56

ms

error rate

asc_sort_with_after_population

0

%

Min Throughput

desc_sort_geonameid

6.01

ops/s

Mean Throughput

desc_sort_geonameid

6.01

ops/s

Median Throughput

desc_sort_geonameid

6.01

ops/s

Max Throughput

desc_sort_geonameid

6.01

ops/s

50th percentile latency

desc_sort_geonameid

9.48206

ms

90th percentile latency

desc_sort_geonameid

10.4281

ms

99th percentile latency

desc_sort_geonameid

10.9411

ms

100th percentile latency

desc_sort_geonameid

11.5985

ms

50th percentile service time

desc_sort_geonameid

8.91875

ms

90th percentile service time

desc_sort_geonameid

9.41848

ms

99th percentile service time

desc_sort_geonameid

10.2572

ms

100th percentile service time

desc_sort_geonameid

10.3711

ms

error rate

desc_sort_geonameid

0

%

Min Throughput

desc_sort_with_after_geonameid

6

ops/s

Mean Throughput

desc_sort_with_after_geonameid

6

ops/s

Median Throughput

desc_sort_with_after_geonameid

6

ops/s

Max Throughput

desc_sort_with_after_geonameid

6

ops/s

50th percentile latency

desc_sort_with_after_geonameid

133.689

ms

90th percentile latency

desc_sort_with_after_geonameid

196.856

ms

99th percentile latency

desc_sort_with_after_geonameid

223.581

ms

100th percentile latency

desc_sort_with_after_geonameid

235.099

ms

50th percentile service time

desc_sort_with_after_geonameid

132.115

ms

90th percentile service time

desc_sort_with_after_geonameid

182.307

ms

99th percentile service time

desc_sort_with_after_geonameid

221.331

ms

100th percentile service time

desc_sort_with_after_geonameid

223.043

ms

error rate

desc_sort_with_after_geonameid

0

%

Min Throughput

asc_sort_geonameid

6.02

ops/s

Mean Throughput

asc_sort_geonameid

6.02

ops/s

Median Throughput

asc_sort_geonameid

6.02

ops/s

Max Throughput

asc_sort_geonameid

6.03

ops/s

50th percentile latency

asc_sort_geonameid

6.92794

ms

90th percentile latency

asc_sort_geonameid

7.68625

ms

99th percentile latency

asc_sort_geonameid

8.04633

ms

100th percentile latency

asc_sort_geonameid

8.13062

ms

50th percentile service time

asc_sort_geonameid

6.30808

ms

90th percentile service time

asc_sort_geonameid

6.6746

ms

99th percentile service time

asc_sort_geonameid

6.92573

ms

100th percentile service time

asc_sort_geonameid

7.03854

ms

error rate

asc_sort_geonameid

0

%

Min Throughput

asc_sort_with_after_geonameid

6

ops/s

Mean Throughput

asc_sort_with_after_geonameid

6

ops/s

Median Throughput

asc_sort_with_after_geonameid

6.01

ops/s

Max Throughput

asc_sort_with_after_geonameid

6.01

ops/s

50th percentile latency

asc_sort_with_after_geonameid

125.278

ms

90th percentile latency

asc_sort_with_after_geonameid

203.164

ms

99th percentile latency

asc_sort_with_after_geonameid

233.588

ms

100th percentile latency

asc_sort_with_after_geonameid

233.875

ms

50th percentile service time

asc_sort_with_after_geonameid

122.861

ms

90th percentile service time

asc_sort_with_after_geonameid

192.424

ms

99th percentile service time

asc_sort_with_after_geonameid

218.462

ms

100th percentile service time

asc_sort_with_after_geonameid

225.712

ms

error rate

asc_sort_with_after_geonameid

0

%

大数据 大数据解决方案 压力测试 ElasticsearchService

0 人点赞