炫意html5
最早CSS3和HTML5移动技术网站之一

MongoDB插入速度突然变得极慢

客户端用的pymongo,程序运行一个周了,之前插入速度都非常快,大概半个小时1千万条,今天突然发现插入速度变得非常慢。

插入目标是goods集合,goods集合现在有1.2亿条数据。

> db.goods.count()

123535156

goods集合的索引如下。

> db.goods.getIndexes();
[
{
“v” : 1,
“key” : {
“_id” : 1
},
“ns” : “shop.goods”,
“name” : “_id_”
},
{
“v” : 1,
“key” : {
“item_id” : 1,
“updated_at” : -1
},
“unique” : true,
“ns” : “shop.goods”,
“name” : “item_id_1_updated_at_-1”
},
{
“v” : 1,
“key” : {
“updated_at” : 1
},
“ns” : “shop.goods”,
“name” : “updated_at_1”
},
{
“v” : 1,
“key” : {
“item_id” : 1
},
“ns” : “shop.goods”,
“name” : “item_id_1”
}
]

机器目前有足够的内存,CPU使用率也不高。

qi@ubuntu :~$ free -m
total       used       free     shared    buffers     cached
Mem:         32162      31965        196          0         37      19739
-/+ buffers/cache:      12189      19972
Swap:         8177         59       8118

Tasks: 198 total,   1 running, 197 sleeping,   0 stopped,   0 zombie
Cpu0  :  2.4%us,  1.3%sy,  0.0%ni, 15.2%id, 80.8%wa,  0.0%hi,  0.3%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.3%us,  0.0%sy,  0.0%ni, 99.3%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  : 20.5%us,  2.0%sy,  0.0%ni, 76.8%id,  0.0%wa,  0.0%hi,  0.7%si,  0.0%st
Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu10 :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu12 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu13 :  1.0%us,  1.0%sy,  0.0%ni, 97.7%id,  0.0%wa,  0.0%hi,  0.3%si,  0.0%st
Cpu14 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu15 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  32934196k total, 30379792k used,  2554404k free,    37928k buffers
Swap:  8374268k total,    61388k used,  8312880k free, 17884864k cached

是什么原因造成的呢?

回答

数据量大了,而且插入时还需要建立索引,固然越来越慢,如果是分片集群,可以把
chunksize设置大点,默认是64,太小会频繁的进行主从交互mangodb 的大数据的情况下,吃内存比较厉害,建议i观察一下内存使用的上升情况

io 使用率呢

炫意HTML5 » MongoDB插入速度突然变得极慢

CSS3教程HTML5教程