Re: Poor RBD performance as LIO iSCSI target

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Chris,

 

I’m doing something very similar to you, however only in a very early stage of testing, but I don’t seem to see the same problem that you are experiencing.

 

My setup is as follows:-

 

1x HP DL360 Server Running ESX 5.1

8x 10K SAS 146GB drives each configured as a RAID 0 and with a separate ESX Datastore on them

4 Virtual Ceph VM’s with 2 OSD’s each residing on above datastores

Ceph VM’s running Ubuntu 14.04 with manually upgraded kernel to 3.16

2x Replicated pool over all OSD’s with a test 50GB RBD

The first 3 VM’s also function as a Pacemaker cluster which exports a HA LIO iSCSI target

 

ESX server then remounts this iSCSI target as a new datastore

Windows 2008 VM running IOmeter sits on this datastore

 

I realise this is a bit of a mess, but it has allowed me to do some basic testing and understand the theory behind gluing Ceph, LIO and pacemaker together. The next stage will be to try and source some more production like hardware to do a more lifelike test.

 

But back to the original point, from this limited hardware I am seeing 200-400MB/s 256kb reads in IOMeter, which is probably about right for the number of disks.

 

I’m afraid I can’t offer much advice at this stage as I’m still fairly early on in understanding Ceph/LIO, but maybe that my performance is not limited suggests that in some combination what we are both trying to achieve is possible. I assume you are using the kernel RBD client? Only other things I can think to try from basic fault finding experience, would be trying the same Kernel version and/or Ubuntu in case there are some minor differences which cause this problem. Of course it maybe that the nested nature of my setup is masking this problem.

 

If you need any configs from my test cluster to compare or any specific tests runs, please let me know.

 

Nick



Nick Fisk
Technical Support Engineer

System Professional Ltd
tel: 01825 830000
mob: 07711377522
fax: 01825 830001
mail: Nick.Fisk@xxxxxxxxxxxxx
web: www.sys-pro.co.uk

IT SUPPORT SERVICES | VIRTUALISATION | STORAGE | BACKUP AND DR | IT CONSULTING

Registered Office:
Wilderness Barns, Wilderness Lane, Hadlow Down, East Sussex, TN22 4HU
Registered in England and Wales.
Company Number: 04754200


Confidentiality: This e-mail and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must take no action based on them, nor must you copy or show them to anyone; please reply to this e-mail and highlight the error.

Security Warning: Please note that this e-mail has been created in the knowledge that Internet e-mail is not a 100% secure communications medium. We advise that you understand and observe this lack of security when e-mailing us.

Viruses: Although we have taken steps to ensure that this e-mail and attachments are free from any virus, we advise that in keeping with good computing practice the recipient should ensure they are actually virus free. Any views expressed in this e-mail message are those of the individual and not necessarily those of the company or any of its subsidiaries.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux