On 05/15/2012 07:21 PM, Idan Kedar wrote: >>> By the way, several weeks I have tried setting up a RAID 5 environment >>> >> with 8 OSDs, 1 mirror and RAID nesting. I then tried cloning and >>> >> compiling the kernel tree over this pNFS-OSD-RAID. The result was that >>> >> otgtd died, and I don't know why. It didn't dump core anywhere I could >>> >> find and the only "log" it has - stdout - didn't give any useful info. >>> >> I was going to inquire about this in a couple of weeks when I need to >>> >> get this environment working, but since this issue came up, maybe we >>> >> can somehow resolve it sooner. >> > >> > >> > 8 OSDs with a mirror ? what was the mkfs.exofs command line you used? > Something along the lines of > # LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --format > --mirrors=1 --group_width=2 --group_depth=2 --dev=/dev/osd0 > --osdname=$(uuid) --dev=/dev/osd1 --osdname=$(uuid) --dev=/dev/osd2 > --osdname=$(uuid) ... > > I don't remember exactly at the moment, but I will bump this thread > when I'll start using RAID again. >> > BTW I don't see a --raid=5 above but a raid5 of --group_width=2 is just a mirror. The minimum for --raid=5 is 3. (I'm not sure if the system checks and optimizes for this) So the above should probably be: # LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 \ --mirrors=2 --group_width=2 --group_depth=1000 \ --dev=/dev/osd0 --format --osdname=$(uuid) \ --dev=/dev/osd1 --format --osdname=$(uuid) \ --dev=/dev/osd2 --format --osdname=$(uuid) \ ... group_depth is how many times to repeat this group until advancing to the next group. The group_count is: group_count = int( (total_devices / mirrors+1) / group_width ) an interesting raid5 configuration with 8 devices mirrors and groups can be: # LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 \ --mirrors=1 --raid=5 --group_width=3 --group_depth=1000 \ --dev=/dev/osd0 --format --osdname=$(uuid) \ --dev=/dev/osd1 --format --osdname=$(uuid) \ --dev=/dev/osd2 --format --osdname=$(uuid) \ Which means each file will have only a single group - 3-out-of-4. but the next file will use another set of 3 devices out of the 4. And I think you need a --stripe_pages= right? Cheers Boaz -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
