说明
本文描述问题及解决方法同样适用于 腾讯云 Elasticsearch Service（ES）。
另外使用到：腾讯云 云服务器（Cloud Virtual Machine，CVM）
环境配置
vespa-fbench 客户端环境
- 版本
Linux环境：Centos 7.9
Python：3.8.7
Pip：pip 20.2.3 from pip (python 3.8)
Java：openjdk version 1.8.0_302 (build 1.8.0_302-b08)
Git：2.7.5
配置
内存：32G
硬盘：SSD云硬盘 100GB
CPU个数：1
CPU核心数：32
Elasticsearch 服务端环境
- 版本
Linux环境：Centos 7.2
Java：openjdk version 11.0.9.1-ga (build 11.0.9.1-ga 1, mixed mode)
Elasticsearch版本：7.10.1（腾讯云 Elasticsearch Service 基础版）
- 配置
节点数量：3
内存：64G
硬盘：增强型SSD云硬盘 1TB
CPU个数：1
CPU核心数：32
CPU型号：AMD EPYC 7K62 48-Core Processor
背景
ES-KNN 插件使⽤了 nmslib 的 HNSW 算法实现来实现近似 k-NN 搜索，本⽂主要介绍使⽤ vespa-fbench 压测工具进行 ES-KNN 的向量检索性能压测。
压测信息
数据集
本篇文档中使用到 GIST 数据集，这个数据集在评估 ANN 的性能和准确性时经常使用，数据集来源 ann-benchmarks。
ES schema
索引信息基于 index.json 调整:
代码语言：javascript复制{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1,
"knn": true
},
"mappings": {
"dynamic": "false",
"_source": {
"enabled": "false"
},
"properties": {
"id": {
"type": "integer"
},
"vector": {
"type": "knn_vector",
"dimension":960
}
}
}
}压测请求示例
共1000条query压测语句，下面是其中一条：
代码语言：javascript复制/doc_knn/_search
/*
* 提示：该行代码过长，系统自动注释不进行高亮。一键复制会移除系统注释
* {"size": 10, "timeout": "15s", "stored_fields": "_none_", "docvalue_fields": ["_id"], "query": {"knn": {"vector": {"vector": [0.011699999682605267, 0.011500000022351742, 0.008700000122189522, 0.009999999776482582, 0.07850000262260437, 0.10000000149011612, 0.07840000092983246, 0.05299999937415123, 0.052400000393390656, 0.08190000057220459, 0.0658000037074089, 0.057999998331069946, 0.01590000092983246, 0.017000000923871994, 0.04610000178217888, 0.02419999986886978, 0.00839999970048666, 0.006399999838322401, 0.007199999876320362, 0.010200000368058681, 0.030400000512599945, 0.06790000200271606, 0.05889999866485596, 0.057100001722574234, 0.0333000011742115, 0.07859999686479568, 0.08919999748468399, 0.04230000078678131, 0.013799999840557575, 0.013299999758601189, 0.028999999165534973, 0.021900000050663948, 0.008999999612569809, 0.012199999764561653, 0.010700000450015068, 0.01080000028014183, 0.026599999517202377, 0.03849999979138374, 0.057100001722574234, 0.052000001072883606, 0.03550000116229057, 0.04879999905824661, 0.06920000165700912, 0.05310000106692314, 0.014399999752640724, 0.014299999922513962, 0.01489999983459711, 0.02500000037252903, 0.017100000753998756, 0.016100000590085983, 0.010599999688565731, 0.03240000084042549, 0.02710000053048134, 0.04580000042915344, 0.05310000106692314, 0.06239999830722809, 0.031599998474121094, 0.06080000102519989, 0.06610000133514404, 0.08129999786615372, 0.01860000006854534, 0.022700000554323196, 0.011599999852478504, 0.03370000049471855, 0.024700000882148743, 0.011800000444054604, 0.010700000450015068, 0.06390000134706497, 0.039500001817941666, 0.040300000458955765, 0.05249999836087227, 0.0957999974489212, 0.05510000139474869, 0.06759999692440033, 0.08579999953508377, 0.17489999532699585, 0.024399999529123306, 0.02810000069439411, 0.008700000122189522, 0.05119999870657921, 0.01489999983459711, 0.00860000029206276, 0.012400000356137753, 0.035599999129772186, 0.03280000016093254, 0.03869999945163727, 0.046300001442432404, 0.048900000751018524, 0.04919999837875366, 0.0640999972820282, 0.07050000131130219, 0.11640000343322754, 0.020600000396370888, 0.016200000420212746, 0.012500000186264515, 0.04019999876618385, 0.013500000350177288, 0.011300000362098217, 0.007400000002235174, 0.011800000444054604, 0.032999999821186066, 0.02979999966919422, 0.0364999994635582, 0.041999999433755875, 0.04410000145435333, 0.051899999380111694, 0.06589999794960022, 0.05270000174641609, 0.013899999670684338, 0.016200000420212746, 0.01510000042617321, 0.02239999920129776, 0.011800000444054604, 0.008799999952316284, 0.007799999788403511, 0.010900000110268593, 0.037300001829862595, 0.07050000131130219, 0.06279999762773514, 0.03909999877214432, 0.02419999986886978, 0.055799998342990875, 0.07209999859333038, 0.04479999840259552, 0.01679999940097332, 0.013399999588727951, 0.03310000151395798, 0.020800000056624413, 0.004100000020116568, 0.008999999612569809, 0.016499999910593033, 0.011699999682605267, 0.09619999676942825, 0.13590000569820404, 0.12150000035762787, 0.08799999952316284, 0.06390000134706497, 0.11739999800920486, 0.09700000286102295, 0.07029999792575836, 0.012600000016391277, 0.021900000050663948, 0.07460000365972519, 0.0421999990940094, 0.003700000001117587, 0.00839999970048666, 0.009999999776482582, 0.013199999928474426, 0.03200000151991844, 0.06129999831318855, 0.07509999722242355, 0.09139999747276306, 0.042899999767541885, 0.0640999972820282, 0.0746999979019165, 0.04659999907016754, 0.008700000122189522, 0.028599999845027924, 0.051500000059604645, 0.027899999171495438, 0.007300000172108412, 0.01489999983459711, 0.02250000089406967, 0.018200000748038292, 0.01810000091791153, 0.01759999990463257, 0.08619999885559082, 0.06400000303983688, 0.027499999850988388, 0.028999999165534973, 0.055799998342990875, 0.060499999672174454, 0.01140000019222498, 0.014499999582767487, 0.02070000022649765, 0.02850000001490116, 0.020800000056624413, 0.019099999219179153, 0.012000000104308128, 0.04500000178813934, 0.023399999365210533, 0.023099999874830246, 0.053300000727176666, 0.08179999887943268, 0.03970000147819519, 0.024299999698996544, 0.03889999911189079, 0.10819999873638153, 0.022099999710917473, 0.030300000682473183, 0.01600000075995922, 0.05339999869465828, 0.02710000053048134, 0.01889999955892563, 0.01640000008046627, 0.08810000121593475, 0.043699998408555984, 0.017000000923871994, 0.028200000524520874, 0.13899999856948853, 0.060100000351667404, 0.03909999877214432, 0.03709999844431877, 0.20309999585151672, 0.03620000183582306, 0.0340999998152256, 0.011900000274181366, 0.07450000196695328, 0.016699999570846558, 0.0052999998442828655, 0.010400000028312206, 0.04560000076889992, 0.022199999541044235, 0.020099999383091927, 0.015599999576807022, 0.04580000042915344, 0.04309999942779541, 0.04019999876618385, 0.04439999908208847, 0.13459999859333038, 0.020600000396370888, 0.019200000911951065, 0.014999999664723873, 0.046799998730421066, 0.010300000198185444, 0.006599999964237213, 0.012000000104308128, 0.017899999395012856, 0.01850000023841858, 0.019500000402331352, 0.04399999976158142, 0.03290000185370445, 0.02979999966919422, 0.0706000030040741, 0.09549999982118607, 0.05810000002384186, 0.009700000286102295, 0.015699999406933784, 0.03189999982714653, 0.0406000018119812, 0.007400000002235174, 0.009800000116229057, 0.010300000198185444, 0.011599999852478504, 0.040699999779462814, 0.08299999684095383, 0.06750000268220901, 0.035599999129772186, 0.03519999980926514, 0.08489999920129776, 0.07400000095367432, 0.057500001043081284, 0.00800000037997961, 0.010499999858438969, 0.050200000405311584, 0.03790000081062317, 0.00989999994635582, 0.021299999207258224, 0.0364999994635582, 0.024700000882148743, 0.04809999838471413, 0.07810000330209732, 0.094200000166893, 0.08399999886751175, 0.04170000180602074, 0.05249999836087227, 0.07289999723434448, 0.06639999896287918, 0.010099999606609344, 0.023099999874830246, 0.0868000015616417, 0.06459999829530716, 0.033799998462200165, 0.02160000056028366, 0.01899999938905239, 0.03700000047683716, 0.027400000020861626, 0.044599998742341995, 0.04349999874830246, 0.10090000182390213, 0.05249999836087227, 0.06769999861717224, 0.057500001043081284, 0.04639999940991402, 0.022700000554323196, 0.04170000180602074, 0.04749999940395355, 0.024000000208616257, 0.06080000102519989, 0.033399999141693115, 0.03680000081658363, 0.0763000026345253, 0.053300000727176666, 0.021900000050663948, 0.06080000102519989, 0.11249999701976776, 0.0674000009894371, 0.021800000220537186, 0.05270000174641609, 0.13120000064373016, 0.05889999866485596, 0.03310000151395798, 0.02160000056028366, 0.06530000269412994, 0.022099999710917473, 0.017400000244379044, 0.030700000002980232, 0.030300000682473183, 0.027300000190734863, 0.029400000348687172, 0.06279999762773514, 0.033799998462200165, 0.04010000079870224, 0.043699998408555984, 0.05900000035762787, 0.07609999924898148, 0.028599999845027924, 0.01889999955892563, 0.04179999977350235, 0.06960000097751617, 0.00800000037997961, 0.007899999618530273, 0.009499999694526196, 0.011099999770522118, 0.08179999887943268, 0.10530000180006027, 0.08699999749660492, 0.05310000106692314, 0.05180000141263008, 0.08860000222921371, 0.07509999722242355, 0.06889999657869339, 0.015799999237060547, 0.018200000748038292, 0.051600001752376556, 0.028599999845027924, 0.006200000178068876, 0.004800000227987766, 0.006800000090152025, 0.009700000286102295, 0.03370000049471855, 0.06939999759197235, 0.07209999859333038, 0.061799999326467514, 0.0364999994635582, 0.08240000158548355, 0.11050000041723251, 0.04879999905824661, 0.013899999670684338, 0.01269999984651804, 0.03139999881386757, 0.020899999886751175, 0.010300000198185444, 0.008500000461935997, 0.009999999776482582, 0.00860000029206276, 0.029200000688433647, 0.03709999844431877, 0.06930000334978104, 0.05209999904036522, 0.031199999153614044, 0.04650000110268593, 0.0835999995470047, 0.04859999939799309, 0.014100000262260437, 0.011900000274181366, 0.014600000344216824, 0.02280000038444996, 0.01600000075995922, 0.012299999594688416, 0.010099999606609344, 0.030300000682473183, 0.027400000020861626, 0.04969999939203262, 0.0706000030040741, 0.0674000009894371, 0.029999999329447746, 0.05730000138282776, 0.05469999834895134, 0.08820000290870667, 0.015699999406933784, 0.020099999383091927, 0.011900000274181366, 0.03530000150203705, 0.02250000089406967, 0.01119999960064888, 0.009700000286102295, 0.05999999865889549, 0.037300001829862595, 0.04410000145435333, 0.05939999967813492, 0.08869999647140503, 0.049400001764297485, 0.06599999964237213, 0.08269999921321869, 0.18400000035762787, 0.023000000044703484, 0.02410000003874302, 0.009800000116229057, 0.05009999871253967, 0.013899999670684338, 0.006399999838322401, 0.01119999960064888, 0.033799998462200165, 0.03150000050663948, 0.03929999843239784, 0.04740000143647194, 0.04089999943971634, 0.04259999841451645, 0.06599999964237213, 0.07320000231266022, 0.125, 0.02239999920129776, 0.01549999974668026, 0.01209999993443489, 0.037300001829862595, 0.01119999960064888, 0.006300000008195639, 0.007499999832361937, 0.01119999960064888, 0.03200000151991844, 0.028300000354647636, 0.04800000041723251, 0.041200000792741776, 0.04190000146627426, 0.05829999968409538, 0.07840000092983246, 0.051600001752376556, 0.013500000350177288, 0.0142000000923872, 0.016899999231100082, 0.019500000402331352, 0.009700000286102295, 0.00570000009611249, 0.005900000222027302, 0.00800000037997961, 0.03819999843835831, 0.07479999959468842, 0.061900001019239426, 0.05050000175833702, 0.02710000053048134, 0.0608999989926815, 0.08449999988079071, 0.05510000139474869, 0.015799999237060547, 0.012000000104308128, 0.03689999878406525, 0.021299999207258224, 0.006200000178068876, 0.014399999752640724, 0.02280000038444996, 0.011300000362098217, 0.10400000214576721, 0.148499995470047, 0.14489999413490295, 0.1062999963760376, 0.06589999794960022, 0.13019999861717224, 0.11670000106096268, 0.09629999846220016, 0.015200000256299973, 0.025599999353289604, 0.08219999819993973, 0.04540000110864639, 0.004699999932199717, 0.00559999980032444, 0.010999999940395355, 0.010099999606609344, 0.03500000014901161, 0.06650000065565109, 0.09539999812841415, 0.11710000038146973, 0.045899998396635056, 0.07039999961853027, 0.09549999982118607, 0.07150000333786011, 0.009700000286102295, 0.028999999165534973, 0.05739999935030937, 0.025100000202655792, 0.008500000461935997, 0.010200000368058681, 0.019999999552965164, 0.014800000004470348, 0.024800000712275505, 0.01850000023841858, 0.09279999881982803, 0.07010000199079514, 0.029999999329447746, 0.033799998462200165, 0.07490000128746033, 0.07490000128746033, 0.01489999983459711, 0.017400000244379044, 0.020999999716877937, 0.03319999948143959, 0.018200000748038292, 0.012199999764561653, 0.008299999870359898, 0.04190000146627426, 0.025200000032782555, 0.024000000208616257, 0.05310000106692314, 0.07620000094175339, 0.048900000751018524, 0.024700000882148743, 0.032999999821186066, 0.1257999986410141, 0.017500000074505806, 0.025599999353289604, 0.013100000098347664, 0.052400000393390656, 0.026100000366568565, 0.01489999983459711, 0.015699999406933784, 0.08609999716281891, 0.04340000078082085, 0.02419999986886978, 0.03620000183582306, 0.1395999938249588, 0.05649999901652336, 0.04270000010728836, 0.04309999942779541, 0.21359999477863312, 0.03449999913573265, 0.025100000202655792, 0.014299999922513962, 0.07569999992847443, 0.014600000344216824, 0.006000000052154064, 0.00930000003427267, 0.041099999099969864, 0.025100000202655792, 0.018699999898672104, 0.03099999949336052, 0.04699999839067459, 0.035100001841783524, 0.05299999937415123, 0.054099999368190765, 0.13439999520778656, 0.016699999570846558, 0.01759999990463257, 0.01489999983459711, 0.04390000179409981, 0.009200000204145908, 0.005499999970197678, 0.01360000018030405, 0.018400000408291817, 0.01860000006854534, 0.02329999953508377, 0.06780000030994415, 0.04050000011920929, 0.024800000712275505, 0.07890000194311142, 0.11249999701976776, 0.0681999996304512, 0.008999999612569809, 0.014399999752640724, 0.033799998462200165, 0.03889999911189079, 0.006599999964237213, 0.012900000438094139, 0.011599999852478504, 0.009100000374019146, 0.041600000113248825, 0.09130000323057175, 0.08060000091791153, 0.04610000178217888, 0.037300001829862595, 0.09560000151395798, 0.07819999754428864, 0.06019999831914902, 0.009200000204145908, 0.013199999928474426, 0.0551999993622303, 0.04190000146627426, 0.009499999694526196, 0.018799999728798866, 0.040300000458955765, 0.028200000524520874, 0.052799999713897705, 0.08269999921321869, 0.11349999904632568, 0.10689999908208847, 0.04320000112056732, 0.05559999868273735, 0.08980000019073486, 0.08179999887943268, 0.011900000274181366, 0.028999999165534973, 0.10509999841451645, 0.07090000063180923, 0.03280000016093254, 0.015799999237060547, 0.019099999219179153, 0.03629999980330467, 0.02800000086426735, 0.045099999755620956, 0.048700001090765, 0.10890000313520432, 0.055799998342990875, 0.07259999960660934, 0.07270000129938126, 0.05249999836087227, 0.02070000022649765, 0.04600000008940697, 0.058400001376867294, 0.025499999523162842, 0.0551999993622303, 0.02630000002682209, 0.03319999948143959, 0.07329999655485153, 0.04749999940395355, 0.025200000032782555, 0.06620000302791595, 0.1120000034570694, 0.062199998646974564, 0.033900000154972076, 0.06599999964237213, 0.1362999975681305, 0.05119999870657921, 0.02759999968111515, 0.024900000542402267, 0.06830000132322311, 0.019899999722838402, 0.01590000092983246, 0.02930000051856041, 0.028599999845027924, 0.02810000069439411, 0.03519999980926514, 0.07850000262260437, 0.03709999844431877, 0.039000000804662704, 0.05090000107884407, 0.07649999856948853, 0.08209999650716782, 0.02500000037252903, 0.015200000256299973, 0.0471000000834465, 0.07530000060796738, 0.007699999958276749, 0.009200000204145908, 0.008500000461935997, 0.008700000122189522, 0.08950000256299973, 0.11569999903440475, 0.11339999735355377, 0.07410000264644623, 0.05290000140666962, 0.1014999970793724, 0.09799999743700027, 0.10660000145435333, 0.013299999758601189, 0.017400000244379044, 0.05990000069141388, 0.028699999675154686, 0.0066999997943639755, 0.008100000210106373, 0.008200000040233135, 0.007699999958276749, 0.03319999948143959, 0.07590000331401825, 0.08829999715089798, 0.08799999952316284, 0.040800001472234726, 0.10760000348091125, 0.1386999934911728, 0.07429999858140945, 0.01080000028014183, 0.012900000438094139, 0.03830000013113022, 0.021400000900030136, 0.006300000008195639, 0.007499999832361937, 0.011599999852478504, 0.007400000002235174, 0.030300000682473183, 0.039000000804662704, 0.08299999684095383, 0.05609999969601631, 0.031099999323487282, 0.06440000236034393, 0.11240000277757645, 0.050200000405311584, 0.011699999682605267, 0.010499999858438969, 0.016599999740719795, 0.022299999371170998, 0.013399999588727951, 0.01140000019222498, 0.012400000356137753, 0.027400000020861626, 0.02590000070631504, 0.05290000140666962, 0.08720000088214874, 0.0575999990105629, 0.02979999966919422, 0.06759999692440033, 0.09369999915361404, 0.08139999955892563, 0.013299999758601189, 0.017799999564886093, 0.01119999960064888, 0.0340999998152256, 0.020800000056624413, 0.01209999993443489, 0.012400000356137753, 0.052799999713897705, 0.029500000178813934, 0.044599998742341995, 0.09889999777078629, 0.066600002348423, 0.043699998408555984, 0.07599999755620956, 0.10930000245571136, 0.1460999995470047, 0.02319999970495701, 0.019300000742077827, 0.01360000018030405, 0.04839999973773956, 0.012000000104308128, 0.007499999832361937, 0.012199999764561653, 0.030899999663233757, 0.029999999329447746, 0.04399999976158142, 0.07959999889135361, 0.040800001472234726, 0.034299999475479126, 0.06960000097751617, 0.09059999883174896, 0.1151999980211258, 0.018300000578165054, 0.01080000028014183, 0.01769999973475933, 0.0364999994635582, 0.010200000368058681, 0.00570000009611249, 0.007899999618530273, 0.010099999606609344, 0.03220000118017197, 0.030899999663233757, 0.07159999758005142, 0.052299998700618744, 0.0340999998152256, 0.06459999829530716, 0.1177000030875206, 0.061400000005960464, 0.011699999682605267, 0.013199999928474426, 0.01899999938905239, 0.0203000009059906, 0.009800000116229057, 0.006599999964237213, 0.0066999997943639755, 0.007600000128149986, 0.04129999876022339, 0.08470000326633453, 0.09740000218153, 0.07699999958276749, 0.02969999983906746, 0.07050000131130219, 0.11559999734163284, 0.08709999918937683, 0.01209999993443489, 0.012299999594688416, 0.04129999876022339, 0.02329999953508377, 0.006099999882280827, 0.012199999764561653, 0.02280000038444996, 0.010999999940395355, 0.12189999967813492, 0.17100000381469727, 0.16410000622272491, 0.14069999754428864, 0.07479999959468842, 0.15860000252723694, 0.15629999339580536, 0.15080000460147858, 0.017100000753998756, 0.034299999475479126, 0.10440000146627426, 0.05310000106692314, 0.005200000014156103, 0.006200000178068876, 0.01209999993443489, 0.011500000022351742, 0.03929999843239784, 0.0697999969124794, 0.10480000078678131, 0.16290000081062317, 0.04520000144839287, 0.09139999747276306, 0.1193000003695488, 0.11640000343322754, 0.007199999876320362, 0.03400000184774399, 0.07320000231266022, 0.030899999663233757, 0.005400000140070915, 0.008500000461935997, 0.01979999989271164, 0.013899999670684338, 0.030700000002980232, 0.02160000056028366, 0.10180000215768814, 0.07959999889135361, 0.03449999913573265, 0.04540000110864639, 0.0957999974489212, 0.0869000032544136, 0.009600000455975533, 0.01889999955892563, 0.026399999856948853, 0.03610000014305115, 0.014700000174343586, 0.011699999682605267, 0.00839999970048666, 0.03440000116825104, 0.032499998807907104, 0.03269999846816063, 0.054999999701976776, 0.06849999725818634, 0.04899999871850014, 0.03020000085234642, 0.04089999943971634, 0.13590000569820404, 0.01590000092983246, 0.016200000420212746, 0.012799999676644802, 0.05389999970793724, 0.015399999916553497, 0.013000000268220901, 0.015200000256299973, 0.06840000301599503, 0.03099999949336052, 0.02889999933540821, 0.049300000071525574, 0.1177000030875206, 0.046799998730421066, 0.0560000017285347, 0.0674000009894371, 0.20250000059604645, 0.026900000870227814, 0.015799999237060547, 0.018400000408291817, 0.06769999861717224, 0.013299999758601189, 0.009700000286102295, 0.009600000455975533, 0.03319999948143959, 0.023499999195337296, 0.024700000882148743, 0.06599999964237213, 0.04699999839067459, 0.022299999371170998, 0.06840000301599503, 0.08169999718666077, 0.12200000137090683, 0.010700000450015068, 0.0142000000923872, 0.01860000006854534, 0.041999999433755875, 0.009200000204145908, 0.006599999964237213, 0.01549999974668026, 0.01730000041425228, 0.02410000003874302, 0.02810000069439411, 0.10459999740123749, 0.07100000232458115, 0.020400000736117363, 0.09690000116825104, 0.15240000188350677, 0.09470000118017197, 0.009499999694526196, 0.013299999758601189, 0.039400000125169754, 0.03909999877214432, 0.007000000216066837, 0.012900000438094139, 0.01360000018030405, 0.00989999994635582, 0.05000000074505806, 0.10570000112056732, 0.09459999948740005, 0.08479999750852585, 0.0430000014603138, 0.1177000030875206, 0.09070000052452087, 0.08169999718666077, 0.008799999952316284, 0.018200000748038292, 0.06419999897480011, 0.04859999939799309, 0.011699999682605267, 0.022700000554323196, 0.05469999834895134, 0.03720000013709068, 0.06459999829530716, 0.093299999833107, 0.15809999406337738, 0.15289999544620514, 0.0494999997317791, 0.05950000137090683, 0.12929999828338623, 0.11240000277757645, 0.011300000362098217, 0.03680000081658363, 0.13169999420642853, 0.07530000060796738, 0.026799999177455902, 0.013000000268220901, 0.021800000220537186, 0.03460000082850456, 0.027499999850988388, 0.04529999941587448, 0.06939999759197235, 0.12280000001192093, 0.05299999937415123, 0.08389999717473984, 0.11110000312328339, 0.06790000200271606, 0.021400000900030136, 0.05640000104904175, 0.07660000026226044, 0.023800000548362732, 0.03959999978542328, 0.01640000008046627, 0.02319999970495701, 0.055799998342990875, 0.03889999911189079, 0.0357000008225441, 0.08150000125169754, 0.10930000245571136, 0.04780000075697899, 0.06700000166893005, 0.10379999876022339, 0.14419999718666077, 0.03739999979734421, 0.021299999207258224, 0.019700000062584877, 0.060100000351667404, 0.0142000000923872, 0.01730000041425228, 0.03280000016093254, 0.02070000022649765, 0.029999999329447746, 0.05119999870657921, 0.11420000344514847, 0.05050000175833702, 0.03370000049471855, 0.07190000265836716, 0.1242000013589859, 0.10620000213384628, 0.023399999365210533, 0.012400000356137753, 0.06030000001192093, 0.08659999817609787], "k": "10"}}}}
*/压测结果
Clients | QPS | Average Latency (ms) | 95P Latency (ms) | Recall@10 |
|---|---|---|---|---|
100 | 1516.34 | 65.89 | 103.30 | 0.9978 |
500 | 1520.57 | 328.76 | 591.70 | |
1000 | 1569.25 | 637.18 | 823.00 | |
1500 | 1843.47 | 813.62 | 1092.90 |
Benchmark 参数
代码语言：javascript复制# Clients 100
/opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 100 -c 0 -i 20 -o /tmp/result.esknn_100.txt 10.0.0.5 9200
# Clients 500
/opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 500 -c 0 -i 20 -o /tmp/result.esknn_500.txt 10.0.0.5 9200
# Clients 1000
/opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 1000 -c 0 -i 20 -o /tmp/result.esknn_1000.txt 10.0.0.5 9200
# Clients 1500
/opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 1500 -c 0 -i 20 -o /tmp/result.esknn_1500.txt 10.0.0.5 9200参数说明
代码语言：javascript复制-s 180：运行时间为180秒，默认为 60，-1 代表永远
-n 1500：1500个客户端进行并发搜索，默认为 10
-c 0：不等待客户端返回结果，直接发送下一个查询请求，默认为 1000，建议留空
-i 20：在前20个查询中忽略延迟（即不计入性能测试结果），以便进行预热，默认为 0
-q：指定查询文件，由make-queries.py生成
-P：使用HTTP POST方法发送请求
-H：指定POST消息体的头信息，为JSON格式特别注意
由于 vespa-fbench 不支持参数或者配置指定http的认证信息，所以当我们的ES集群有身份认证时，则需要在压测命令的请求头中加入认证信息。
代码语言：javascript复制/opt/vespa/bin/vespa-fbench -P -H "Content-Type:application/json" -H "Authorization: Basic $(echo -n 'elastic:changeme' | base64)" -q data/elastic/knn_queries.txt -s 180 -n 1500 -c 1000 -i 20 -o /tmp/result.esknn_1500.txt 10.0.0.5 9200压测用例
1. 安装压测工具 vespa-fbench
代码语言：javascript复制# 添加yum源仓库
[root@centos ~]# yum-config-manager --add-repo
https://copr.fedorainfracloud.org/coprs/g/vespa/vespa/repo/epel-7/group_vespa-vespa-epel-7.repo
[root@centos ~]# yum -y install epel-release centos-release-scl
# 安装vespa
[root@centos ~]# yum -y install vespa安装完之后，会在/opt/vespa/bin目录下面成可执行文件，我们需要的执行命令是vespa-fbench
2. 克隆项目 dense-vector-ranking-performance
我们需要在ES集群中创建需要压测的索引并导入数据集，以及生成压测的请求
代码语言：javascript复制[root@centos ~]# git clone https://github.com/jobergum/dense-vector-ranking-performance.git
Cloning into 'dense-vector-ranking-performance'...
remote: Enumerating objects: 149, done.
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 149 (delta 0), reused 0 (delta 0), pack-reused 147
Receiving objects: 100% (149/149), 532.09 MiB | 725.00 KiB/s, done.
Resolving deltas: 100% (56/56), done.
[root@centos ~]# cd dense-vector-ranking-performance
[root@centos dense-vector-ranking-performance]# ll
total 52
drwxr-xr-x 5 root root 4096 May 10 13:45 bin
drwxr-xr-x 5 root root 4096 May 10 13:45 config
drwxr-xr-x 5 root root 4096 May 10 13:45 data
-rw-r--r-- 1 root root 187 May 10 13:45 Dockerfile.elastic
-rw-r--r-- 1 root root 344 May 10 13:45 Dockerfile.opendistroforelasticsearch
-rw-r--r-- 1 root root 102 May 10 13:45 Dockerfile.vespa
-rw-r--r-- 1 root root 11357 May 10 13:45 LICENSE
-rw-r--r-- 1 root root 15017 May 10 13:45 README.md
[root@centos ~]#3. 准备数据集 GIST
这里我们需要申请一台海外的机器，否则该数据集可能下载耗时1天以上。
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# wget http://ann-benchmarks.com/gist-960-euclidean.hdf5下载完成之后，我们通过 coscmd 的 upload 功能，将数据集上传到 cos 对象存储，便于拷贝到国内的服务器上
代码语言：javascript复制[root@VM-16-2-centos ~]# coscmd upload gist-960-euclidean.hdf5 /
Upload gist-960-euclidean.hdf5 => cos://knn-1253240642/gist-960-euclidean.hdf5
100%|███████████████████████████████████████| 3.58G/3.58G [00:23<00:00, 162MB/s]
[root@centos ~]# 4. 修改配置
dense-vector-ranking-performance默认使用的是本地环境进行配置的生成，而我们需要对现有的服务器进行压测，所以需要修改配置以达到目的。
需要创建2个文件，以及修改3个文件。
4.1 创建文件 config/elastic/index_knn.json
定义压测索引的属性：
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# cat config/elastic/index_knn.json
{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1,
"knn": true
},
"mappings": {
"dynamic": "false",
"_source": {
"enabled": "false"
},
"properties": {
"id": {
"type": "integer"
},
"vector": {
"type": "knn_vector",
"dimension":960
}
}
}
}
[root@centos dense-vector-ranking-performance]# 4.2 创建文件 create_knn-index.sh
引用索引创建属性进行索引创建：
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# cat create_knn-index.sh
#!/bin/sh
curl -s -X PUT "http://10.0.0.16:9200/doc_knn?pretty" -H "Content-Type:application/json" -d @config/elastic/index_knn.json
[root@centos dense-vector-ranking-performance]# 4.3 修改 bin/make-feed.py
数据集导入：
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# cat bin/make-feed.py
import h5py
import sys
import concurrent.futures
import requests
file= sys.argv[1]
train = h5py.File(file, 'r')['train']
def feed_to_es_and_vespa(data):
docid,vector = data
vector = vector.tolist()
vespa_body = {
"fields": {
'vector': {
'values': vector
},
'id': docid
}
}
es_body={
'id': docid,
'vector': vector
}
response = requests.post('http://10.0.0.5:9200/doc_knn/_doc/%i' %docid, json=es_body)
response.raise_for_status()
nthreads=32
with concurrent.futures.ThreadPoolExecutor(max_workers=nthreads) as executor:
futures = [executor.submit(feed_to_es_and_vespa,data) for data in enumerate(train)]
for result in concurrent.futures.as_completed(futures):
pass
[root@centos dense-vector-ranking-performance]# 4.4 修改 bin/make-queries.py
生成用于压测的query请求体文件：
代码语言：javascript复制[root@centos dense-vector-ranking-performance-master]# cat bin/make-queries.py
import numpy as np
import json
import h5py
import sys
file= sys.argv[1]
test= h5py.File(file, 'r')['test']
esknn_queries = open('data/elastic/knn_queries.txt', 'w')
for v in test:
query_vector = v.tolist()
esknn_script_query = {
'knn': {
'vector': {
'vector': query_vector,
'k': '10'
}
}
}
esknn_body={
'size': 10,
'timeout': '15s',
'stored_fields': '_none_',
'docvalue_fields': ['_id'],
'query': esknn_script_query
}
esknn_queries.write('/doc_knn/_searchn')
esknn_queries.write(json.dumps(esknn_body) 'n')
[root@centos dense-vector-ranking-performance]# 4.4 修改 bin/check-recall.py
召回测试配置修改：
代码语言：javascript复制[root@VM-10-4-centos dense-vector-ranking-performance]# cat bin/check-recall.py
import numpy as np
import json
import h5py
import sys
import requests
file= sys.argv[1]
data = h5py.File(file, 'r')
def get_elastic_result(query_vector):
esknn_script_query = {
'knn': {
'vector': {
'vector': query_vector,
'k': '10'
}
}
}
esknn_body={
'size': 10,
'timeout': '15s',
'query': esknn_script_query
}
response = requests.post('http://10.0.0.5:9200/doc_knn/_search', json=esknn_body)
response.raise_for_status()
hits=[]
for h in response.json()['hits']['hits']:
id = h['_id']
score = h['_score']
hits.append(int(id))
return hits
def compute_recall(real_neighbors,computed_neighbors, n=10):
real_neighbors = real_neighbors[0:n]
recalled = 0
for r_n in real_neighbors:
if r_n in computed_neighbors:
recalled=recalled 1
return recalled/n
average_recall_elastic = []
average_recall_vespa = []
average_recall_opendistroforelastic = []
for i,vector in enumerate(data['test'][0:1000]):
real_neighbors = data['neighbors'][i]
distances = data['distances'][i]
vector=vector.tolist()
computed_neighbors_es = get_elastic_result(vector)
recall_elastic = compute_recall(real_neighbors, computed_neighbors_es)
average_recall_elastic.append(recall_elastic)
print('Average recall Elastic KNN = %f' % np.average(average_recall_elastic))
5. 导入数据集并生成压测请求体文件
5.1 创建索引
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# bash create_knn-index.sh
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "doc_knn"
}5.2 导入数据集
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# python3 ./bin/make-feed.py gist-960-euclidean.hdf55.3 segment合并
数据集导入完成之后，进行一次forcemerge，便于压测
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# curl -XPOST -s '10.0.0.5:9200/doc_knn/_forcemerge?max_num_segments=1'5.4 生成压测请求体文件
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# python3 ./bin/make-queries.py gist-960-euclidean.hdf56. 运行benchmark
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# /opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 100 -c 1000 -i 20 -o /tmp/result.esknn_100.txt 10.0.0.5 92007. 运行召回测试
代码语言：javascript复制[root@centos dense-vector-ranking-performance]# python3 bin/check-recall.py gist-960-euclidean.hdf5附录
压测明细：
代码语言：javascript复制# Clients 100
/opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 100 -c 0 -i 20 -o /tmp/result.esknn_100.txt 10.0.0.5 9200
# Clients 500
/opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 500 -c 0 -i 20 -o /tmp/result.esknn_500.txt 10.0.0.5 9200
# Clients 1000
/opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 1000 -c 0 -i 20 -o /tmp/result.esknn_1000.txt 10.0.0.5 9200
# Clients 1500
/opt/vespa/bin/vespa-fbench -P -H Content-Type:application/json -q data/elastic/knn_queries.txt -s 180 -n 1500 -c 0 -i 20 -o /tmp/result.esknn_1500.txt 10.0.0.5 9200
============================100============================
Starting clients...
[dummydate]: PROGRESS: vespa-fbench: Seconds left 180
[dummydate]: PROGRESS: vespa-fbench: Seconds left 120
[dummydate]: PROGRESS: vespa-fbench: Seconds left 60
Stopping clients
Clients stopped.
Clients Joined.
*** HTTP keep-alive statistics ***
connection reuse count -- 272922
***************** Benchmark Summary *****************
clients: 100
ran for: 180 seconds
cycle time: 0 ms
lower response limit: 0 bytes
skipped requests: 0
failed requests: 0
successful requests: 271022
cycles not held: 271022
minimum response time: 6.62 ms
maximum response time: 209.89 ms
average response time: 65.89 ms
25 percentile: 50.30 ms
50 percentile: 65.20 ms
75 percentile: 80.70 ms
90 percentile: 94.60 ms
95 percentile: 103.30 ms
98 percentile: 113.40 ms
99 percentile: 120.70 ms
99.5 percentile: 128.40 ms
99.6 percentile: 130.80 ms
99.7 percentile: 133.70 ms
99.8 percentile: 138.10 ms
99.9 percentile: 145.30 ms
actual query rate: 1516.34 Q/s
utilization: 99.91 %
zero hit queries: 0
zero hit percentage: 0.00 %
http request status breakdown:
200 : 273022
============================500============================
Starting clients...
[dummydate]: PROGRESS: vespa-fbench: Seconds left 180
[dummydate]: PROGRESS: vespa-fbench: Seconds left 120
[dummydate]: PROGRESS: vespa-fbench: Seconds left 60
Stopping clients
Clients stopped.
Clients Joined.
*** HTTP keep-alive statistics ***
connection reuse count -- 274650
***************** Benchmark Summary *****************
clients: 500
ran for: 180 seconds
cycle time: 0 ms
lower response limit: 0 bytes
skipped requests: 0
failed requests: 0
successful requests: 265150
cycles not held: 265150
minimum response time: 6.95 ms
maximum response time: 748.57 ms
average response time: 328.76 ms
25 percentile: 220.30 ms
50 percentile: 382.90 ms
75 percentile: 429.50 ms
90 percentile: 540.00 ms
95 percentile: 591.70 ms
98 percentile: 629.30 ms
99 percentile: 646.90 ms
99.5 percentile: 659.80 ms
99.6 percentile: 663.40 ms
99.7 percentile: 667.80 ms
99.8 percentile: 674.30 ms
99.9 percentile: 682.19 ms
actual query rate: 1520.57 Q/s
utilization: 99.98 %
zero hit queries: 0
zero hit percentage: 0.00 %
http request status breakdown:
200 : 275150
============================1000============================
Starting clients...
[dummydate]: PROGRESS: vespa-fbench: Seconds left 180
[dummydate]: PROGRESS: vespa-fbench: Seconds left 120
[dummydate]: PROGRESS: vespa-fbench: Seconds left 60
Stopping clients
Clients stopped.
Clients Joined.
*** HTTP keep-alive statistics ***
connection reuse count -- 284687
***************** Benchmark Summary *****************
clients: 1000
ran for: 180 seconds
cycle time: 0 ms
lower response limit: 0 bytes
skipped requests: 0
failed requests: 0
successful requests: 265687
cycles not held: 265687
minimum response time: 60.49 ms
maximum response time: 1127.86 ms
average response time: 637.18 ms
25 percentile: 562.80 ms
50 percentile: 631.50 ms
75 percentile: 707.50 ms
90 percentile: 779.20 ms
95 percentile: 823.00 ms
98 percentile: 874.70 ms
99 percentile: 906.20 ms
99.5 percentile: 932.30 ms
99.6 percentile: 939.20 ms
99.7 percentile: 947.00 ms
99.8 percentile: 958.20 ms
99.9 percentile: 974.73 ms
actual query rate: 1569.25 Q/s
utilization: 99.99 %
zero hit queries: 0
zero hit percentage: 0.00 %
http request status breakdown:
200 : 285240
429 : 447
============================1500============================
Starting clients...
[dummydate]: PROGRESS: vespa-fbench: Seconds left 180
[dummydate]: PROGRESS: vespa-fbench: Seconds left 120
[dummydate]: PROGRESS: vespa-fbench: Seconds left 60
Stopping clients
Clients stopped.
Clients Joined.
*** HTTP keep-alive statistics ***
connection reuse count -- 334485
***************** Benchmark Summary *****************
clients: 1500
ran for: 180 seconds
cycle time: 0 ms
lower response limit: 0 bytes
skipped requests: 0
failed requests: 0
successful requests: 305985
cycles not held: 305985
minimum response time: 6.26 ms
maximum response time: 1826.55 ms
average response time: 813.62 ms
25 percentile: 790.00 ms
50 percentile: 882.80 ms
75 percentile: 957.10 ms
90 percentile: 1030.10 ms
95 percentile: 1092.90 ms
98 percentile: 1199.20 ms
99 percentile: 1272.70 ms
99.5 percentile: 1336.80 ms
99.6 percentile: 1355.01 ms
99.7 percentile: 1378.50 ms
99.8 percentile: 1401.40 ms
99.9 percentile: 1445.50 ms
actual query rate: 1843.47 Q/s
utilization: 99.99 %
zero hit queries: 0
zero hit percentage: 0.00 %
http request status breakdown:
200 : 281304
429 : 54681
