mongodb分片部署(未启用密码认证)
环境:
OS:Centos 7
mongodb:4.4.22
拓扑结构:
s1分片(副本集):
192.168.104:29001 分片服务器1
192.168.106:29001 分片服务器2
192.168.107:29001 分片服务器3
s2分片(副本集):
192.168.104:29002 分片服务器1
192.168.106:29002 分片服务器2
192.168.107:29002 分片服务器3
s3分片(副本集):
192.168.104:29003 分片服务器1
192.168.106:29003 分片服务器2
192.168.107:29003 分片服务器3
配置服务器副本集
192.168.104:28001
192.168.106:28002
192.168.107:28002
路由服务器
192.168.108:30001
说明:
3.4以上的版本,分片服务器和配置服务器都需要配置成副本集的模式
####################部署s1副本集分片服务器######################
1.下载相应的版本
https://www.mongodb.com/download-center/community
我这里下载的是mongodb-linux-x86_64-rhel70-4.4.22.tgz
2.在每个分片服务器上创建如下目录
每个s1分片服务器上都执行
[root@test services]# mkdir -p /usr/local/services
[root@test services]# mkdir -p /home/middle/mongodb_s1/data
[root@test services]# mkdir -p /home/middle/mongodb_s1/log
[root@test services]# mkdir -p /home/middle/mongodb_s1/key
[root@test services]# mkdir -p /home/middle/mongodb_s1/conf
[root@test services]# mkdir -p /home/middle/mongodb_s1/run
3.在每个分片服务器上进行安装
每个s1分片服务器上都执行
[root@test soft]# tar -xvf mongodb-linux-x86_64-rhel70-4.4.22.tgz
[root@test soft]# mv mongodb-linux-x86_64-rhel70-4.4.22 /usr/local/services/mongodb_s1
4.产生秘钥验证(先不做)
在其中一个机器上创建秘钥文件,我这里是在192.168.1.104:29001
[root@test key]# cd /home/middle/mongodb_s1/key
[root@test key]# openssl rand -base64 741 >>keyfile
[root@test key]# chmod 700 keyfile
加个keyfile拷贝到另外的两个节点相应的目录
scp keyfile root@192.168.1.106:/home/middle/mongodb/key/
scp keyfile root@192.168.1.107:/home/middle/mongodb/key/
5.生成日志文件(配置文件中指定了,提前创建)
每个s1分片服务器上都执行
[root@test key]#echo>/home/middle/mongodb_s1/log/mongodb.log
6.创建配置文件 mongo.cnf
每个s1分片服务器上都执行,需要修改相应的ip和端口
vi /home/middle/mongodb_s1/conf/mongo.cnf
port=29001
fork=true
dbpath=/home/middle/mongodb_s1/data
logpath=/home/middle/mongodb_s1/log/mongodb.log
pidfilepath=/home/middle/mongodb_s1/run/29001.pid
bind_ip=192.168.1.104,127.0.0.1
logappend=true
shardsvr=true
replSet=s1
oplogSize=16384
logRotate=reopen
##keyFile=/home/middle/mongodb_s1/key/keyfile
##auth=true
7.启动s1的每个分片服务器
每个s1分片服务器上都执行
/usr/local/services/mongodb_s1/bin/mongod -f /home/middle/mongodb_s1/conf/mongo.cnf
8.初始化s1
在s1副本集的其中一台机器上执行即可
[root@localhost bin]# /usr/local/services/mongodb_s1/bin/mongo 192.168.1.104:29001
use admin
config={_id:'s1',members:[{_id:0,host:'192.168.1.104:29001'},{_id:1,host:'192.168.1.106:29001'},{_id:2,host:'192.168.1.107:29001'}]}
rs.initiate(config)
9.查看副本集s1集群状态
rs.status()
rs.conf()
另外两个副本集也安装上面的步骤进行操作,将s1替换成s2,s3,同时注意ip和端口的修改.
登录查看集群情况
/usr/local/services/mongodb_s2/bin/mongo 192.168.1.104:29002
/usr/local/services/mongodb_s3/bin/mongo 192.168.1.104:29003
####################部署配置服务器######################
1.在每个分片服务器上创建如下目录
每个副本集服务器上都执行
[root@test services]# mkdir -p /usr/local/services
[root@test services]# mkdir -p /home/middle/mongo_config/data
[root@test services]# mkdir -p /home/middle/mongo_config/log
[root@test services]# mkdir -p /home/middle/mongo_config/key
[root@test services]# mkdir -p /home/middle/mongo_config/conf
[root@test services]# mkdir -p /home/middle/mongo_config/run
2.解压安装
每个副本集服务器上都执行
[root@test soft]# tar -xvf mongodb-linux-x86_64-rhel70-4.4.22.tgz
[root@test soft]# mv mongodb-linux-x86_64-rhel70-4.4.22 /usr/local/services/mongo_config
3.生成日志文件
每个副本集服务器上都执行
[root@localhost soft]#echo>/home/middle/mongo_config/log/mongodb.log
4.创建配置文件
在conf目录下创建配置文件,每个副本集服务器上都执行,注意修改ip地址
vi /home/middle/mongo_config/conf/mongo.cnf
[root@localhost conf]# more mongo.cnf
port=28001
fork=true
dbpath=/home/middle/mongo_config/data
logpath=/home/middle/mongo_config/log/mongodb.log
pidfilepath=/home/middle/mongo_config/run/28001.pid
bind_ip=192.168.1.104,127.0.0.1
logappend=true
oplogSize=16384
logRotate=reopen
configsvr=true
replSet=configrs
5.启动
每个副本集服务器上都执行
/usr/local/services/mongo_config/bin/mongod -f /home/middle/mongo_config/conf/mongo.cnf
6.初始化副本集
在副本集的其中一台机器上执行即可
[root@localhost bin]# /usr/local/services/mongodb_s1/bin/mongo 192.168.1.104:28001
use admin
config={_id:'configrs',members:[{_id:0,host:'192.168.1.104:28001'},{_id:1,host:'192.168.1.106:28001'},{_id:2,host:'192.168.1.107:28001'}]}
rs.initiate(config)
7.查看副本集状态
rs.status()
rs.conf()
8.配置服务器副本集下的库和表
configrs:PRIMARY> show dbs
admin 0.000GB
config 0.002GB
local 0.002GB
configrs:PRIMARY> use config
switched to db config
configrs:PRIMARY> show tables
actionlog
changelog
chunks
collections
databases
image_collection
lockpings
locks
migrations
mongos
shards
system.indexBuilds
tags
transactions
version
####################部署路由服务器######################
1.在每个分片服务器上创建如下目录
每个分片服务器上都执行
[root@test services]# mkdir -p /usr/local/services
[root@test services]# mkdir -p /home/middle/mongo_router/data
[root@test services]# mkdir -p /home/middle/mongo_router/log
[root@test services]# mkdir -p /home/middle/mongo_router/key
[root@test services]# mkdir -p /home/middle/mongo_router/conf
[root@test services]# mkdir -p /home/middle/mongo_router/run
2.解压安装
[root@test soft]# tar -xvf mongodb-linux-x86_64-rhel70-4.4.22.tgz
[root@test soft]# mv mongodb-linux-x86_64-rhel70-4.4.22 /usr/local/services/mongo_router
3.生成日志文件
[root@localhost soft]#echo>/home/middle/mongo_router/log/mongodb.log
4.创建配置文件
在conf目录下创建配置文件,mongo.cnf
vi /home/middle/mongo_router/conf/mongo.cnf
[root@localhost conf]# more mongo.cnf
port=30001
fork=true
logpath=/home/middle/mongo_router/log/mongodb.log
pidfilepath=/home/middle/mongo_router/run/30001.pid
configdb=configrs/192.168.1.104:28001,192.168.1.106:28001,192.168.1.107:28001
bind_ip=192.168.1.108,127.0.0.1
5.启动
/usr/local/services/mongo_router/bin/mongos -f /home/middle/mongo_router/conf/mongo.cnf
注意这里是使用mongos命令启动,而不是mongod
6.路由服务器下的库表
mongos> show databases
admin 0.000GB
config 0.004GB
hxl 0.010GB ##后面步骤新增
mongos> use config
switched to db config
mongos> show tables
actionlog
changelog
chunks
collections
databases
image_collection
lockpings
locks
migrations
mongos
shards
system.indexBuilds
tags
transactions
version
################添加分片服务器#############################
1.添加分片服务器
目前搭建了mongodb配置服务器、路由服务器，各个分片服务器，不过应用程序连接到mongos路由服务器并不能使用分片机制，还需要在程序里设置分片配置，让分片生效。
登录路由服务器(192.168.1.108)
[root@pxc01 bin]# /usr/local/services/mongo_router/bin/mongo 192.168.1.108:30001
mongos> use admin
switched to db admin
#串联路由服务器与分配副本集
sh.addShard("s1/192.168.1.104:29001,192.168.1.106:29001,192.168.1.107:29001")
sh.addShard("s2/192.168.1.104:29002,192.168.1.106:29002,192.168.1.107:29002")
sh.addShard("s3/192.168.1.104:29003,192.168.1.106:29003,192.168.1.107:29003")
#查看集群状态
sh.status()
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("64b64e0705045e89b67bb0f6")
}
shards:
{ "_id" : "s1", "host" : "s1/192.168.1.104:29001,192.168.1.106:29001,192.168.1.107:29001", "state" : 1 }
{ "_id" : "s2", "host" : "s2/192.168.1.104:29002,192.168.1.106:29002,192.168.1.107:29002", "state" : 1 }
{ "_id" : "s3", "host" : "s3/192.168.1.104:29003,192.168.1.106:29003,192.168.1.107:29003", "state" : 1 }
active mongoses:
"4.4.22" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
3.设置分片存储的数据库,这个时候hxl数据库是不存在的
[root@pxc01 bin]# /usr/local/services/mongo_router/bin/mongo 192.168.1.108:30001
mongos> use admin
mongos> db.runCommand({enablesharding:"hxl" })
{
"ok" : 1,
"operationTime" : Timestamp(1689673064, 4),
"$clusterTime" : {
"clusterTime" : Timestamp(1689673064, 5),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
mongos> db.runCommand({"shardcollection":"hxl.person","key":{"_id":"hashed"}})
{
"collectionsharded" : "hxl.person",
"collectionUUID" : UUID("753575b3-2974-4223-b007-5bbf6f4efd88"),
"ok" : 1,
"operationTime" : Timestamp(1689673137, 14),
"$clusterTime" : {
"clusterTime" : Timestamp(1689673137, 14),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
在路由服务器上执行该命令后，后台会自动在每个分片服务器上创建hxl数据库和person集合
写入数据
mongos> use hxl
switched to db hxl
mongos> for (var i=0;i<100000;i++){db.person.insert({"name":"hxl"+i,"age":i})}
4.数据核查
登录路由服务器
[root@pxc01 bin]# /usr/local/services/mongo_router/bin/mongo 192.168.1.108:30001
mongos> use hxl
switched to db hxl
mongos> db.person.find().count()
100000
分别登录s1,s2,s3 三个分片查看数据分布情况
s1:
[root@localhost soft]# /usr/local/services/mongodb_s1/bin/mongo 192.168.1.104:29001
s1:SECONDARY> use hxl
s1:PRIMARY> db.person.find().count()
33197
s2:
[root@localhost soft]#/usr/local/services/mongodb_s1/bin/mongo 192.168.1.104:29002
s2:SECONDARY> rs.secondaryOk()
s2:SECONDARY> use hxl
s2:SECONDARY> db.person.find().count()
33500
s3:
[root@localhost soft]#/usr/local/services/mongodb_s1/bin/mongo 192.168.1.104:29003
s3:SECONDARY> rs.secondaryOk()
s3:SECONDARY> use hxl
s3:SECONDARY> db.person.find().count()
33303
33197 + 33500 + 33303=100000,可以看到数据均匀的分布到3个分片了,但是发现写入速度很慢.
##############################日常维护#####################################
1.启动顺序:
配置服务器-->分片服务器器-->路由服务器
启动配置服务器
192.168.1.104
192.168.1.106
192.168.1.107
/usr/local/services/mongo_config/bin/mongod -f /home/middle/mongo_config/conf/mongo.cnf
启动分片s1
192.168.1.104
192.168.1.106
192.168.1.107
/usr/local/services/mongodb_s1/bin/mongod -f /home/middle/mongodb_s1/conf/mongo.cnf
启动分片s2
192.168.1.104
192.168.1.106
192.168.1.107
/usr/local/services/mongodb_s2/bin/mongod -f /home/middle/mongodb_s2/conf/mongo.cnf
启动分片s3
192.168.1.104
192.168.1.106
192.168.1.107
/usr/local/services/mongodb_s3/bin/mongod -f /home/middle/mongodb_s3/conf/mongo.cnf
启动路由服务器
192.168.1.108
/usr/local/services/mongo_router/bin/mongos -f /home/middle/mongo_router/conf/mongo.cnf