Re: Initdb --data-checksums by default

Alex Ignatov <a.ignatov@xxxxxxxxxxxxxx> · Wed, 20 Apr 2016 12:02:31 +0300



    On 20.04.2016 11:40, Andreas Joseph
      Krogh wrote:

    
      På onsdag 20. april 2016 kl. 10:33:14, skrev Alex Ignatov
        <a.ignatov@xxxxxxxxxxxxxx>:
      
        
          On 20.04.2016 11:29, Devrim Gündüz wrote:

          > Hi,

          >

          > On Wed, 2016-04-20 at 10:43 +0300, Alex Ignatov wrote:

          >> Today in Big Data epoch silent data corruption
          becoming more and more

          >> issue to afraid of. With uncorrectable read error
          rate ~ 10^-15  on

          >> multiterabyte disk bit rot is the real issue.

          >> I think that today checksumming data  must be
          mandatory  set by default.

          >> Only if someone doesn't care about his data he can
          manually turn this

          >> option off.

          >>

          >> What do you think about defaulting --data-checksums
          in initdb?

          > I think this should be discussed in -hackers, right?

          >

          > Regards,

          May be you right but i want to know what people think about it
          before

          i'll write to hackers.
      
       
      -1 on changing the default.
       
      10^15 ~= 1000 TB, which isn't very common yet. Those having
        it probably are aware of the risk and have enabled checksums
        already.
       
      
        --
          Andreas
                Joseph Krogh
          CTO
              / Partner - Visena AS
          Mobile:
              +47 909 56 963
          andreas@xxxxxxxxxx
          www.visena.com
          
        
    It is per bit not bytes. So it is ~100 TB. We working with some
    enterprise who have WALs creation rate ~ 4GB per min - so it is only
    max 100 days before you get bit rotted and have probability to get
    silent data corruption.

    Also don't forget that it is theoretical limit and Google tells us
    that HDD and SSD is not as reliable as manufactures tell. So this
    10^-15 can easily be much higher.

    -- 
Alex Ignatov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company