On 21/05/2013, at 18.19, Gregory Farnum <greg@xxxxxxxxxxx> wrote: > On Tue, May 21, 2013 at 9:01 AM, Emil Renner Berthing <ceph@xxxxxxxx> wrote: >> On 21 May 2013 17:55, Gregory Farnum <greg@xxxxxxxxxxx> wrote: >>> On Tue, May 21, 2013 at 8:44 AM, Emil Renner Berthing <ceph@xxxxxxxx> wrote: >>>> Hi Greg, >>>> >>>> Here are some more stats on our servers: >>>> - each server has 64GB ram, >>>> - there are 12 OSDs pr. server, >>>> - each OSD uses around 1.5 GB of memory, >>>> - we have 18432 PGs, >>>> - around 5 to 10 MB writes/s is written to each OSD and almost no reads (yet). >>> >>> What interface are you writing with? How many OSD servers are there? >> >> We're using librados and there are 132 OSDs so far. > > Okay, so the allocation is happening in the depths of LevelDB — maybe > the issue is there somewhere. Are you doing anything weird with omap, > snapshots, or xattrs? I can help; No, we are not using omap, snaps or weird stuff with xattrs. We are storing objects of sizes from few KB to GBs. Also, we have a quirk in the application design right now, which means that we store an object, write it again under a new name and delete the original object. - if that is of any potential value to finding this. -- Anders -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
