mongo内存中排序可能会导致ram不足报错

2022-11-06 09:39:33 浏览数 (1)

如果对查询结果较大的数据量进行排序,超过了一定的ram大小,那么就会提示ram不足。

我这里提示 maximum 33554432 bytes 也就是 大约32M的情况下,就已经开始提示了。所以建议,如果一次性查出了很大的数据量,就不要在内存中排序了。

那么有什么好的解决方法吗?

在MongoDB中,排序操作,可以通过从索引中按照索引的顺序获取文档的方式,来保证结果的有序性。

如果MongoDB的查询计划器(planner) 没法从索引中得到排序顺序,那么它就需要在内存中对结果排序。

相比于“不用索引的排序”操作,用索引会有更好的性能。

注意,关键是:不用索引的排序操作,会在用了超过32MB内存时终止,也就是说MongoDB只能支持32MB的非索引排序 。

如果数据量很大,比如我目前的应用场景下,都是千万级数据量,而且飞速增加,达亿级…

代码语言:javascript复制
Query failed with error code 96 and error message 'Executor error during find command :: caused by :: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.' on server 127.0.0.1:12000; nested exception is com.mongodb.MongoQueryException: Query failed with error code 96 and error message 'Executor error during find command :: caused by :: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.' on server 127.0.0.1:12000
org.springframework.data.mongodb.UncategorizedMongoDbException: Query failed with error code 96 and error message 'Executor error during find command :: caused by :: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.' on server 127.0.0.1:12000; nested exception is com.mongodb.MongoQueryException: Query failed with error code 96 and error message 'Executor error during find command :: caused by :: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.' on server 127.0.0.1:12000
        at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:138)
        at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2793)
        at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2698)
        at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2418)
        at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2401)
        at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:829)
    
        at com.xxl.job.core.thread.JobThread.run(JobThread.java:152)
Caused by: com.mongodb.MongoQueryException: Query failed with error code 96 and error message 'Executor error during find command :: caused by :: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.' on server 127.0.0.1:12000
        at com.mongodb.operation.FindOperation$1.call(FindOperation.java:706)
        at com.mongodb.operation.FindOperation$1.call(FindOperation.java:695)
        at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:462)
        at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:406)
        at com.mongodb.operation.FindOperation.execute(FindOperation.java:695)

0 人点赞