Re: multiple journals on SSD

George Shuklin <george.shuklin@xxxxxxxxx> · Tue, 12 Jul 2016 16:16:09 +0300



    Yes, linear io speed was concern during
      benchmark. I can not predict how much linear IO would be generated
      by clients (compare to IOPS) so we going to balance HDD-OSD per
      SSD according to real usage. If users would generate too much
      random IO, we will raise HDD/SSD ratio, if they would generate
      more linear-write load, we will reduce that number. I plan to do
      it by reserving space for 'more HDD' or 'more SSD' in planned
      servers - they will go to production with ~50% slot utilization.

      
      My main concern is that random IO for OSD includes not only
      writes, but reads too, and cold random read will slower HDD
      performance significantly. On my previous experience, any weekly
      cronjob on server with backup (or just 'find /') cause bad spikes
      of cold read, and that drastically diminish HDD performance.

      
      (TL;DR; I don't belive we ever would have conditions when HDD can
      give few dozens of MB/s of writing).

      
      Thank you for advice.

      
      On 07/12/2016 04:03 PM, Vincent Godin wrote:

    
              Hello.

                
                I've been testing Intel 3500 as journal store for few
                HDD-based OSD. I

                stumble on issues with multiple partitions (>4) and
                UDEV (sda5, sda6,etc

                sometime do not appear after partition creation). And
                I'm thinking that

                partition is not that useful for OSD management, because
                linux do no

                allow partition rereading with it contains used volumes.

                
                So my question: How you store many journals on SSD? My
                initial thoughts:

                
                1)  filesystem with filebased journals

                2) LVM with volumes

                
                Anything else? Best practice?

              
                P.S. I've done benchmarking: 3500 can support up to 16
                10k-RPM HDD.

                
              Hello,

              
            I would like to advertise you not using 1 SSD for 16 HDD.
            Ceph journal is not only a journal but a write cache during
            operation. I had that kind of configuration with 1 SSD for
            20  SATA HDD. With a Ceph bench, i notice that my rate whas
            limited between 350 and 400 MB/s. In fact, a iostat show me
            that my SSD was 100% utilised with a rate of 350-400 MB/s. 

            
          If you consider that a SATA HDD can have a max average rate of
          100 MB/s, you need to configure one SSD (which can rate till
          400 MB/s) for 4 SATA HDD

          
        Vincent

      
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com