Re: crc32 for erasure code

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 23/09/2013 18:59, Gregory Farnum wrote:
> On Mon, Sep 23, 2013 at 1:34 AM, Loic Dachary <loic@xxxxxxxxxxx> wrote:
>> Hi,
>>
>> Unless I'm mistaken, ceph_crc32() is currently used in master via the crc32c() method of bufferlist to:
>>
>> * encode_with_checksum/decode_with_checksum a PGLog entry
>> * Message::decode_message/Message::encode_message a message via calc_*_crc
>> * FileJournal::do_read_entry/FileJournal::prepare_single_write a journal entry
>> * for information in the report monitory command ( Monitor.cc )
>>
>> Erasure coded chunks ( i.e. files ) will need checksums. Should this be implemented as an optional feature in ceph/src/os/FileStore.{h,cc} ? If the underlying filesystem does not provide this feature, FileStore would call ceph_crc32 each time the object is modified. A verification method would be exposed and used when scrubbing erasure coded pools.
> 
> You mean should checksums be optional in the FileStore, or should we
> provide a plugin framework for using things other than crc32, or...?
> :)

Not really. I was under the impression that crc32 is good enough. I'm not sure where ( in the code path ) it should be used for erasure coded pools. In the FileStore ? Or in the erasure code PG to set an attribute of the object ? The FileStore seems to be more sensible but ... I'm not sure hence the mail ;-)

> -Greg
> Software Engineer #42 @ http://inktank.com | http://ceph.com
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.

Attachment: signature.asc
Description: OpenPGP digital signature


[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux