//
MongoDB运维与开发（六）---MongoDB集群(5)
//
关于MongoDB的集群运维，之前的文章已经说了很多内容了，这块儿知识点比较多，由于是每天抽空写文章，所以每天能说到的点有限，慢慢一点一点搞懂它，今天我们来看MongDB集群里面的分片相关知识。
关于分片和副本集的概念，之前的文章中也讲过，上一篇文章中讲了MongoS和ConfigDB的配置方法，今天来看如何在MongoS上添加Mongod分片。
01
初始化分片
通常情况下，我们配置好了mongos和configDB之后，需要在Mongos上面配置分片节点，配置分片节点的之前，需要有几项准备工作：
1、启动分片节点副本集
2、在mongos的配置文件中写清除configDB的地址
3、保证mongos可以同时访问configDB和分片节点
然后，我们通过下面的命令来配置初始化添加一个分片节点副本集：
代码语言：javascript复制use admin
db.runCommand({addShard:"sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020"})
可以看到，我们添加了一个名字叫做sharding_yeyz的副本集，它作为Mongos的分片加入到这个集群中，它包含27018、27019、27020三个mongod实例，架构图如下：
02
在将要分片的集合上面创建索引
代码语言：javascript复制mongos> use test
switched to db test
mongos> db.test0.ensureIndex({_id:"hashed"})
{
"raw" : {
"sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020" : {
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1
}
},
"ok" : 1,
"operationTime" : Timestamp(1605690613, 4),
"$clusterTime" : {
"clusterTime" : Timestamp(1605690613, 4),
"signature" : {
"hash" : BinData(0,"9ThfDRTC/X/GSuLbe8E7TjXLP2I="),
"keyId" : NumberLong("6894922308364795934")
}
}
}
如上面的命令，我们在test库下面的test0集合上的_id字段上创建hash索引，也就是：
db.test0.ensureIndex({_id:"hashed"})
03
对db和collection启用分片
首先我们通过下面的命令，对test库启用分片：
代码语言：javascript复制mongos> sh.enableSharding("test")
{
"ok" : 1,
"operationTime" : Timestamp(1605690781, 3),
"$clusterTime" : {
"clusterTime" : Timestamp(1605690781, 3),
"signature" : {
"hash" : BinData(0,"YrrrniMbOp8Vfjkf8cYulX n9oA="),
"keyId" : NumberLong("6894922308364795934")
}
}
}
可以看到，命令sh.enableSharding对库test启用了分片。
然后对test0这个collection启用分片：
代码语言：javascript复制mongos> sh.shardCollection("test.test0",{_id:"hashed"})
{
"collectionsharded" : "test.test0",
"collectionUUID" : UUID("a1bafe79-c74e-4454-ae98-ce1b175cda2c"),
"ok" : 1,
"operationTime" : Timestamp(1605690845, 10),
"$clusterTime" : {
"clusterTime" : Timestamp(1605690845, 10),
"signature" : {
"hash" : BinData(0,"PIttL1zPmNcjjGZ1BpBIZVrhT6I="),
"keyId" : NumberLong("6894922308364795934")
}
}
}
04
插入数据，观察分片状态
我们循环插入number=1、2、3~1000的数字，并观察当前分片状态如下：
代码语言：javascript复制### 循环程序，插入i=1、2、3、4...1000到test.test0这个集合
mongos> for (var i=1 ;i<=1000 ; i ){ db.test1.insert({"number":i})}
WriteResult({ "nInserted" : 1 })
### 使用printShardingStatus查看分片状态
mongos> db.printShardingStatus()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5fafaf4f5785d9965548f687")
}
shards: # 涉及的分片只有sharding_yeyz
{ "_id" : "sharding_yeyz", "host" : "sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020", "state" : 1 }
active mongoses:
"4.0.6" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 1
Last reported error: Could not find host matching read preference { mode: "primary" } for set sharding_yeyz
Time of Reported error: Wed Nov 18 2020 17:08:14 GMT 0800 (CST)
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
sharding_yeyz 1
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : sharding_yeyz Timestamp(1, 0)
{ "_id" : "test", "primary" : "sharding_yeyz", "partitioned" : true, "version" : { "uuid" : UUID("f1d54510-2675-45e1-b61b-2eb0fe015f2d"), "lastMod" : 1 } }
test.test0
shard key: { "_id" : "hashed" }
unique: false
balancing: true
chunks: # 两个数据chunk都在sharding_yeyz上
sharding_yeyz 2
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong(0) } on : sharding_yeyz Timestamp(1, 0)
{ "_id" : NumberLong(0) } -->> { "_id" : { "$maxKey" : 1 } } on : sharding_yeyz Timestamp(1, 1)
从分片状态可以看到，当前只存在一个分片sharding_yeyz,所有的数据都在该分片上，然后我们查询一下数据：
代码语言：javascript复制sharding_yeyz:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99a"), "number" : 1 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99b"), "number" : 2 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99c"), "number" : 3 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99d"), "number" : 4 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99e"), "number" : 5 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99f"), "number" : 6 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a0"), "number" : 7 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a1"), "number" : 8 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a2"), "number" : 9 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a3"), "number" : 10 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a4"), "number" : 11 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a5"), "number" : 12 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a6"), "number" : 13 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a7"), "number" : 14 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a8"), "number" : 15 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a9"), "number" : 16 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9aa"), "number" : 17 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ab"), "number" : 18 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ac"), "number" : 19 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ad"), "number" : 20 }
Type "it" for more
sharding_yeyz:PRIMARY> db.test0.count()
1000
可以看到所有数据都在，数据总条数是1000条，和我们设想的结果一致。
05
加入一个新的分片
此时我们加入一个新的分片副本集，名字叫做sharding_yeyz1,它由27024、27025、27026三个mongod实例组成，如下：
代码语言：javascript复制db.runCommand({addShard:"sharding_yeyz1/127.0.0.1:27024,127.0.0.1:27025,127.0.0.1:27026"})
{
"shardAdded" : "sharding_yeyz1",
"ok" : 1,
"operationTime" : Timestamp(1605697822, 2),
"$clusterTime" : {
"clusterTime" : Timestamp(1605697822, 3),
"signature" : {
"hash" : BinData(0,"wpspZ ILVa7R5Zybhks 3da9N4g="),
"keyId" : NumberLong("6894922308364795934")
}
}
}
数据的架构变成了：
此时我们重新对test库下面的test1集合启用分片，并插入1~1000的数据，如下：
代码语言：javascript复制启用分片
sh.shardCollection("test.test1",{_id:"hashed"})
插入数据
for (var i=1 ;i<=1000 ; i ){ db.test1.insert({"number":i})}
来看sharding_yeyz和sharding_yeyz1上test.test1这个集合的查询结果：
代码语言：javascript复制sharding_yeyz:PRIMARY> db.test1.find()
{ "_id" : ObjectId("5fb50645eebfcce3b111e552"), "number" : 1 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e553"), "number" : 2 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e554"), "number" : 3 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e555"), "number" : 4 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e556"), "number" : 5 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e557"), "number" : 6 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55b"), "number" : 10 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55e"), "number" : 13 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e563"), "number" : 18 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e565"), "number" : 20 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e566"), "number" : 21 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e568"), "number" : 23 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e569"), "number" : 24 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56a"), "number" : 25 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56b"), "number" : 26 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56d"), "number" : 28 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56e"), "number" : 29 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e573"), "number" : 34 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e576"), "number" : 37 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e577"), "number" : 38 }
Type "it" for more
sharding_yeyz:PRIMARY> db.test1.count()
519
--------------------
sharding_yeyz1:PRIMARY> db.test1.find()
{ "_id" : ObjectId("5fb50645eebfcce3b111e558"), "number" : 7 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e559"), "number" : 8 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55a"), "number" : 9 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55c"), "number" : 11 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55d"), "number" : 12 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55f"), "number" : 14 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e560"), "number" : 15 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e561"), "number" : 16 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e562"), "number" : 17 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e564"), "number" : 19 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e567"), "number" : 22 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56c"), "number" : 27 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56f"), "number" : 30 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e570"), "number" : 31 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e571"), "number" : 32 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e572"), "number" : 33 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e574"), "number" : 35 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e575"), "number" : 36 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e57a"), "number" : 41 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e57b"), "number" : 42 }
Type "it" for more
sharding_yeyz1:PRIMARY> db.test1.count()
481
可以看到，sharding_yeyz保存有519条数据，而sharding_yeyz1保存有481条数据，并不是严格的按照50%的比例分配的，而且number的序号，也不是严格按照奇数偶数来分的。
还有一个问题？已经分配了的test0集合，1000条记录原本都在sharding_yeyz这个分片上，加入了sharding_yeyz1，这1000条记录会不会重新分配到第二个分片上呢？我们来看实际结果：
代码语言：javascript复制------------原始分片------------
sharding_yeyz:PRIMARY> db.test0.count()
492
sharding_yeyz:PRIMARY>
sharding_yeyz:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99b"), "number" : 2 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99d"), "number" : 4 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99f"), "number" : 6 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a1"), "number" : 8 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a2"), "number" : 9 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a3"), "number" : 10 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a4"), "number" : 11 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a5"), "number" : 12 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a6"), "number" : 13 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a7"), "number" : 14 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ae"), "number" : 21 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b0"), "number" : 23 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b2"), "number" : 25 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b3"), "number" : 26 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b4"), "number" : 27 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b5"), "number" : 28 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b6"), "number" : 29 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b7"), "number" : 30 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b9"), "number" : 32 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bd"), "number" : 36 }
Type "it" for more
----------------新增分片------------
sharding_yeyz1:PRIMARY> db.test0.count()
508
sharding_yeyz1:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99a"), "number" : 1 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99c"), "number" : 3 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99e"), "number" : 5 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a0"), "number" : 7 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a8"), "number" : 15 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a9"), "number" : 16 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9aa"), "number" : 17 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ab"), "number" : 18 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ac"), "number" : 19 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ad"), "number" : 20 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9af"), "number" : 22 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b1"), "number" : 24 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b8"), "number" : 31 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ba"), "number" : 33 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bb"), "number" : 34 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bc"), "number" : 35 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9be"), "number" : 37 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bf"), "number" : 38 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9c1"), "number" : 40 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9c4"), "number" : 43 }
从结果中不难看出来，原来test0这个集合上面的1000条记录，也都从分片1上相对均匀的分配到了分片2上面，而分片1上的数据进行了清理，不再是之前的1000条。
到这里，大家对MongoDB的分片应该有了如下认识：
1、分片是自动的
2、分片不是绝对均匀的
后续的文章，我们会说明分片受哪些因素的影响。
今天就到这里吧。
