好玩的ES--第三篇之过滤查询，整合SpringBoot

2022-05-10 15:38:30 浏览数 (1)

好玩的ES--第三篇之过滤查询，整合SpringBoot

  • 过滤查询
    • 过滤查询
    • 使用
    • 类型
      • term 、 terms Filter
      • ranage filter
      • exists filter
      • ids filter
  • 整合应用
    • 引入依赖
    • 配置客户端
    • 客户端对象
      • ElasticsearchOperations
        • 索引文档
        • 删除文档
        • 查询文档
        • 更新文档
        • 删除所有
        • 查询所有
      • RestHighLevelClient
        • 创建索引映射
        • 索引文档
        • 更新文档
        • 删除文档
        • 基于 id 查询文档
        • 查询所有
        • 综合查询
        • 过滤查询
        • 思路扩展

好玩的ES—第一篇之安装和基本CRUD

好玩的ES–第二篇之高级查询，索引原理和分词器

过滤查询

过滤查询

过滤查询，其实准确来说，ES中的查询操作分为2种: 查询(query)和过滤(filter)。查询即是之前提到的query查询，它 (查询)默认会计算每个返回文档的得分，然后根据得分排序。而过滤(filter)只会筛选出符合的文档，并不计算 得分，而且它可以缓存文档 。所以，单从性能考虑，过滤比查询更快。 换句话说过滤适合在大范围筛选数据，而查询则适合精确匹配数据。一般应用时， 应先使用过滤操作过滤数据， 然后使用查询匹配数据。

使用

代码语言：javascript复制
GET /ems/emp/_search
{
  "query": {
    "bool": {
      "must": [
        {"match_all": {}} //查询条件
      ],
      "filter": {....} //过滤条件
  }
}
  • 注意:
    • 在执行 filter 和 query 时,先执行 filter 在执行 query
    • Elasticsearch会自动缓存经常使用的过滤器，以加快性能。

类型

常见过滤类型有: term 、 terms 、ranage、exists、ids等filter。

term 、 terms Filter

代码语言：javascript复制
GET /ems/emp/_search   # 使用term过滤
{
  "query": {
    "bool": {
      "must": [
        {"term": {
          "name": {
            "value": "小黑"
          }
        }}
      ],
      "filter": {
        "term": {
          "content":"框架"
        }
      }
    }
  }
}
GET /dangdang/book/_search  #使用terms过滤
{
  "query": {
    "bool": {
      "must": [
        {"term": {
          "name": {
            "value": "中国"
          }
        }}
      ],
      "filter": {
        "terms": {
          "content":[
              "科技",
              "声音"
            ]
        }
      }
    }
  }
}

ranage filter

代码语言：javascript复制
GET /ems/emp/_search
{
  "query": {
    "bool": {
      "must": [
        {"term": {
          "name": {
            "value": "中国"
          }
        }}
      ],
      "filter": {
        "range": {
          "age": {
            "gte": 7,
            "lte": 20
          }
        }
      }
    }
  }
}

exists filter

过滤存在指定字段,获取字段不为空的索引记录使用

代码语言：javascript复制
GET /ems/emp/_search
{
  "query": {
    "bool": {
      "must": [
        {"term": {
          "name": {
            "value": "中国"
          }
        }}
      ],
      "filter": {
        "exists": {
          "field":"aaa"
        }
      }
    }
  }
}

ids filter

过滤含有指定字段的索引记录

代码语言：javascript复制
GET /ems/emp/_search
{
  "query": {
    "bool": {
      "must": [
        {"term": {
          "name": {
            "value": "中国"
          }
        }}
      ],
      "filter": {
        "ids": {
          "values": ["1","2","3"]
        }
      }
    }
  }
}

整合应用

引入依赖

代码语言：javascript复制
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

配置客户端

代码语言：javascript复制
@Data
@Configuration
public class RestClientConfig extends AbstractElasticsearchConfiguration {
    @Value("${es.host}")
    private String ES_HOST;
    @Override
    @Bean
    public RestHighLevelClient elasticsearchClient() {
        final ClientConfiguration clientConfiguration = ClientConfiguration.builder()
                .connectedTo(ES_HOST)
                .build();
        return RestClients.create(clientConfiguration).rest();
    }
}

es默认开放了两个端口进行访问，一个是9200的rest方式访问，一个是9300的tcp方式访问，这里推荐使用9200rest方式的访问

客户端对象

  • ElasticsearchOperations
  • RestHighLevelClient 推荐

ElasticsearchOperations

  • 特点: 始终使用面向对象方式操作 ES
    • 索引: 用来存放相似文档集合
    • 映射: 用来决定放入文档的每个字段以什么样方式录入到 ES 中 字段类型 分词器…
    • 文档: 可以被索引最小单元 json 数据格式

相关注解

代码语言：javascript复制
@Data
@Document(indexName = "products", createIndex = true)
public class Product {
    @Id
    private Integer id;
    @Field(type = FieldType.Keyword)
    private String title;
    @Field(type = FieldType.Float)
    private Double price;
    @Field(type = FieldType.Text)
    private String description;
}
代码语言：javascript复制
//1. @Document(indexName = "products", createIndex = true) 用在类上 作用:代表一个对象为一个文档
		-- indexName属性: 创建索引的名称
    -- createIndex属性: 是否创建索引
//2. @Id 用在属性上  作用:将对象id字段与ES中文档的_id对应
//3. @Field(type = FieldType.Keyword) 用在属性上 作用:用来描述属性在ES中存储类型以及分词情况
    -- type: 用来指定字段类型
索引文档
代码语言：javascript复制
    @Test
    public void testCreate() throws IOException {
        Product product = new Product();
        product.setId(1); //存在id指定id  不存在id自动生成id
        product.setTitle("怡宝矿泉水");
        product.setPrice(129.11);
        product.setDescription("我们喜欢喝矿泉水....");
        //文档不存在会创建文档,文档存在会更新文档
        elasticsearchOperations.save(product);
    }
删除文档
代码语言：javascript复制
    @Test
    public void testDelete() {
        Product product = new Product();
        product.setId(1);
        String delete = elasticsearchOperations.delete(product);
        System.out.println(delete);
    }
查询文档
代码语言：javascript复制
    @Test
    public void testGet() {
        Product product = elasticsearchOperations.get("1", Product.class);
        System.out.println(product);
    }
更新文档
代码语言：javascript复制
 @Test
public void testUpdate() {
  Product product = new Product();
  product.setId(1);
  product.setTitle("怡宝矿泉水");
  product.setPrice(129.11);
  product.setDescription("我们喜欢喝矿泉水,你们喜欢吗....");
  elasticsearchOperations.save(product);//不存在添加,存在更新
}
删除所有
代码语言：javascript复制
@Test
public void testDeleteAll() {
  elasticsearchOperations.delete(Query.findAll(), Product.class);
}
查询所有
代码语言：javascript复制
@Test
public void testFindAll() {
  SearchHits<Product> productSearchHits = elasticsearchOperations.search(Query.findAll(), Product.class);
  productSearchHits.forEach(productSearchHit -> {
    System.out.println("id: "   productSearchHit.getId());
    System.out.println("score: "   productSearchHit.getScore());
    Product product = productSearchHit.getContent();
    System.out.println("product: "   product);
  });
}

RestHighLevelClient

创建索引映射
代码语言：javascript复制
    @Test
    public void testCreateIndex() throws IOException {
        CreateIndexRequest createIndexRequest = new CreateIndexRequest("fruit");
        createIndexRequest.mapping("{n"  
                "    "properties": {n"  
                "      "title":{n"  
                "        "type": "keyword"n"  
                "      },n"  
                "      "price":{n"  
                "        "type": "double"n"  
                "      },n"  
                "      "created_at":{n"  
                "        "type": "date"n"  
                "      },n"  
                "      "description":{n"  
                "        "type": "text"n"  
                "      }n"  
                "    }n"  
                "  }n" , XContentType.JSON);
        CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(createIndexRequest, RequestOptions.DEFAULT);
        System.out.println(createIndexResponse.isAcknowledged());
        restHighLevelClient.close();
    }
索引文档
代码语言：javascript复制
    @Test
    public void testIndex() throws IOException {
        IndexRequest indexRequest = new IndexRequest("fruit");
        indexRequest.source("{n"  
                "          "id" : 1,n"  
                "          "title" : "蓝月亮",n"  
                "          "price" : 123.23,n"  
                "          "description" : "这个洗衣液非常不错哦！"n"  
                "        }",XContentType.JSON);
        IndexResponse index = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
        System.out.println(index.status());
    }
更新文档
代码语言：javascript复制
    @Test
    public void testUpdate() throws IOException {
        UpdateRequest updateRequest = new UpdateRequest("fruit","qJ0R9XwBD3J1IW494-Om");
        updateRequest.doc("{"title":"好月亮"}",XContentType.JSON);
        UpdateResponse update = restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);
        System.out.println(update.status());
    }
删除文档
代码语言：javascript复制
    @Test
    public void testDelete() throws IOException {
        DeleteRequest deleteRequest = new DeleteRequest("fruit","1");
        DeleteResponse delete = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
        System.out.println(delete.status());
    }
基于 id 查询文档
代码语言：javascript复制
    @Test
    public void testGet() throws IOException {
        GetRequest getRequest = new GetRequest("fruit","aPbmV38BvtuRfHsTIvNo");
        GetResponse getResponse = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT);
        System.out.println(getResponse.getSourceAsString());
    }
查询所有
代码语言：javascript复制
    public void commonExampleSearch(String indice, QueryBuilder queryBuilder) throws IOException {
        SearchRequest searchRequest = new SearchRequest(indice);
        SearchSourceBuilder sourceBuilder=new SearchSourceBuilder();
        sourceBuilder.query(queryBuilder);
        searchRequest.source(sourceBuilder);
        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println("总记录数: " searchResponse.getHits().getTotalHits().value);
        System.out.println("最大得分: " searchResponse.getHits().getMaxScore());
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

    @Test
    public void testSearch() throws IOException {
        String indice="fruit";
        //查询所有
        commonExampleSearch(indice,QueryBuilders.matchAllQuery());
       //term查询
        commonExampleSearch(indice,QueryBuilders.termQuery("description","不错哦!"));
       //prefix查询
       commonExampleSearch(indice,QueryBuilders.prefixQuery("description","这个"));
       //通配符查询
        commonExampleSearch(indice,QueryBuilders.wildcardQuery("title","好*"));
       //ids查询--多id查询
       commonExampleSearch(indice,QueryBuilders.idsQuery().addIds("1","2"));
       //多字段查询
        commonExampleSearch(indice,QueryBuilders.multiMatchQuery("不错","title","description"));
     }
综合查询
代码语言：javascript复制
    @Test
    public void testSearch1() throws IOException {
        SearchRequest searchRequest = new SearchRequest("fruit");
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder
                //分页查询
                .from(0)//起始位置 start=(page-1)*size
                .size(2)//每页显示条数,默认返回10条
                //指定排序字段,参数一:根据哪个字段进行排序,参数二:排序方式
                .sort("price", SortOrder.DESC)
                //返回的结果中排除或者包含哪些字段
                //参数1:包含的字段数组
                //参数2:排除字段数组
                .fetchSource(new String[]{"title"},new String[]{})
                //高亮设置
                .highlighter(new HighlightBuilder()
                        //高亮显示的字段
                        .field("description")
                        //多字段高亮开启
                        .requireFieldMatch(false)
                        //自定义高亮html标签
                        .preTags("<span style='color:red;'>").postTags("</span>"))
                //查询
                .query(QueryBuilders.termQuery("description","错"));
        searchRequest.source(sourceBuilder);
        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println("总条数: " searchResponse.getHits().getTotalHits().value);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
            //显示当前查询结果中出现的高亮字段
            Map<String, HighlightField> highlightFields = hit.getHighlightFields();
            highlightFields.forEach((k,v)-> System.out.println("key: " k   " value: " v.fragments()[0]));
        }
    }
过滤查询
代码语言：javascript复制
    /**
     * query: 精确查询,查询计算文档得分,并根据文档得分进行返回
     * filter query: 过滤查询,用来在大量数据中筛选出本地查询相关数据,不会计算文档得分,经常使用filter query结果进行缓存
     * 注意: 一旦使用query和filterQuery es优先执行filter query 然后再执行 query
     */
    @Test
    public void testFilterQuery() throws IOException {
      SearchRequest searchRequest=new SearchRequest("fruit");
      SearchSourceBuilder sourceBuilder=new SearchSourceBuilder();
      sourceBuilder.query(QueryBuilders.termQuery("description","不错"))
              //指定过滤条件
              .postFilter(QueryBuilders.idsQuery().addIds("1","2","3"));
    
      searchRequest.source(sourceBuilder);

        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);

        System.out.println("符合条件的总数为: " searchResponse.getHits().getTotalHits().value);
    }
思路扩展

ElasticsearchOperations面向对象的查询方式，有其优点所在，那么我们能否将其和RestHighLevelClient 进行互补呢 ？

看下面的例子:

代码语言：javascript复制
@AllArgsConstructor
@NoArgsConstructor
@Builder
@Data
public class Fruit implements Serializable {
    private String title;
    private Double price;
    private Date create_at;
    private String description;
}
代码语言：javascript复制
/**
 * @author 大忽悠
 * @create 2022/3/5 11:34
 */
public class AllTest extends EsApplicationTests{
  ObjectMapper objectMapper=new ObjectMapper();
    /**
     * 添加文档
     */
    @Test
    public void addIndice() throws IOException {
        Fruit fruit = Fruit.builder().id(5).title("大忽悠").price(520.521)
                .description("大忽悠喜欢小朋友")
                .build();
        IndexRequest indexRequest=new IndexRequest("fruit");
        indexRequest.id(fruit.getId().toString()).source(objectMapper.writeValueAsString(fruit),XContentType.JSON);
        IndexResponse index = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
        System.out.println(index.status());
    }

    public void commonExampleSearch(String indice, QueryBuilder queryBuilder) throws IOException {
        SearchRequest searchRequest = new SearchRequest(indice);
        SearchSourceBuilder sourceBuilder=new SearchSourceBuilder();
        sourceBuilder.query(queryBuilder);
        searchRequest.source(sourceBuilder);
        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println("总记录数: " searchResponse.getHits().getTotalHits().value);
        System.out.println("最大得分: " searchResponse.getHits().getMaxScore());
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println("返回的结果为: " hit.getSourceAsString());
            //JSON格式转换
            Fruit fruit = objectMapper.readValue(hit.getSourceAsString(), Fruit.class);
            System.out.println("得到的结果为: " fruit);
        }
    }

    /**
     * 查询全部
     */
    @Test
   public void searchAll() throws IOException {
       String indice="fruit";
       //查询所有
       commonExampleSearch(indice, QueryBuilders.matchAllQuery());
   }
}

注意: 两者的区别，因此我们在上传文档的时候，一定要通过objectMapper.writeValueAsString(fruit)的方式先转换为JSON串的原因

代码语言：javascript复制
        Fruit fruit = Fruit.builder().id(5).title("大忽悠").price(520.521)
                .description("大忽悠喜欢小朋友")
                .build();

        System.out.println(objectMapper.writeValueAsString(fruit));
        System.out.println(fruit);
es ElasticsearchService 编程算法

0 人点赞