Re: HUGE slowdown when doing dpkg with ext4 over nbd

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Dec 7, 2016, at 10:58 AM, Andreas Dilger <adilger@xxxxxxxxx> wrote:
> 
> On Dec 7, 2016, at 2:52 AM, Renaud Mariana <rmariana@xxxxxxxxxx> wrote:
>> 
>> Here are my answers, hope it will help solve this issue, thanks.
>> 
>> Recap:
>> dpkg kibana on ext4 over a nbd device takes 10 minutes
>> with xfs it's only 30s.
>> with ext4 no extends only 30s.
>> 
>> 
>> kernels :
>> 4.5.7 has this issue as older kernel like 4.4.34
>> The issue is also when nbd client & server run on same host
>> 
>> 
>> How small are the files?
>> here is the histogram of file sizes : http://pasteboard.co/6HC3nKyk2.png
>> We can see 5000 files around 512 Bytes.
> 
> Definitely there is no value to use fallocate for 512-byte files, or any
> of the files that can be written in a single write() syscall.  I'd expect
> any reasonable tool to be using a write buffer of at least 2-4MB these
> days to get good performance, so writes below the buffer size shouldn't
> use fallocate() at all.
> 
>> dpkg using fallocate() ?
>> Yes, there are 16044 calls by the same process
>> what are these uninitialized extents ?
> 
> Uninitialized extents are preallocated ranges of a file on disk that will
> read back as zero, but are not necessarily zero-filled at allocation time.
> For large files that are written randomly (or written slowly and may have
> contention from other writers) fallocate() + uninitialized extents will
> preallocate the space for the file so that it is (largely) contiguous on
> disk and overwrites will not result in random block allocation.

It would probably be worthwhile to file a ticket in the Debian BTS:
https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;package=dpkg

with your information, so that there is something to submit a patch against.

Cheers, Andreas





Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail


[Index of Archives]     [Reiser Filesystem Development]     [Ceph FS]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite National Park]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Media]

  Powered by Linux