On Wed, May 16, 2012 at 3:15 PM, Boaz Harrosh <bharrosh@xxxxxxxxxxx> wrote: > On 05/15/2012 07:21 PM, Idan Kedar wrote: > >>>> By the way, several weeks I have tried setting up a RAID 5 environment >>>> >> with 8 OSDs, 1 mirror and RAID nesting. I then tried cloning and >>>> >> compiling the kernel tree over this pNFS-OSD-RAID. The result was that >>>> >> otgtd died, and I don't know why. It didn't dump core anywhere I could >>>> >> find and the only "log" it has - stdout - didn't give any useful info. >>>> >> I was going to inquire about this in a couple of weeks when I need to >>>> >> get this environment working, but since this issue came up, maybe we >>>> >> can somehow resolve it sooner. >>> > >>> > >>> > 8 OSDs with a mirror ? what was the mkfs.exofs command line you used? >> Something along the lines of >> # LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --format >> --mirrors=1 --group_width=2 --group_depth=2 --dev=/dev/osd0 >> --osdname=$(uuid) --dev=/dev/osd1 --osdname=$(uuid) --dev=/dev/osd2 >> --osdname=$(uuid) ... >> >> I don't remember exactly at the moment, but I will bump this thread >> when I'll start using RAID again. >>> > > > > BTW I don't see a --raid=5 above but a raid5 of --group_width=2 is just > a mirror. The minimum for --raid=5 is 3. (I'm not sure if > the system checks and optimizes for this) > > So the above should probably be: > > # LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 \ > --mirrors=2 --group_width=2 --group_depth=1000 \ > --dev=/dev/osd0 --format --osdname=$(uuid) \ > --dev=/dev/osd1 --format --osdname=$(uuid) \ > --dev=/dev/osd2 --format --osdname=$(uuid) \ > ... > > group_depth is how many times to repeat this group until > advancing to the next group. > > The group_count is: > group_count = int( (total_devices / mirrors+1) / group_width ) > > an interesting raid5 configuration with 8 devices mirrors and groups > can be: > > # LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 \ > --mirrors=1 --raid=5 --group_width=3 --group_depth=1000 \ > --dev=/dev/osd0 --format --osdname=$(uuid) \ > --dev=/dev/osd1 --format --osdname=$(uuid) \ > --dev=/dev/osd2 --format --osdname=$(uuid) \ > > Which means each file will have only a single group - 3-out-of-4. > but the next file will use another set of 3 devices out of the 4. > > And I think you need a --stripe_pages= right? > > Cheers > Boaz As I said, I don't remember the exact line, so you are probably right. I will be testing the object RAID functionality, so the more "interesting" - the better. I'll use the last one you suggested, and I would appreciate any input about which topologies give the RAID code a hard time. thanks, idank -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
