Re: rsync very slow

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, 2023-10-30 at 22:58 +1100, fedora@xxxxxxxxxxxxxx wrote:
> On 30/10/2023 22.41, Patrick O'Callaghan wrote:
> > On Mon, 2023-10-30 at 22:35 +1100, fedora@xxxxxxxxxxxxxx wrote:
> > > On 30/10/2023 21.40, Patrick O'Callaghan wrote:
> > > > On Mon, 2023-10-30 at 18:22 +1100, fedora@xxxxxxxxxxxxxx wrote:
> > > > > Should I blame rsync (or the way I use it)?
> > > > > 
> > > > > Next, I will do a plain 'cp -a' between the original source
> > > > > (sdh
> > > > > on
> > > > > /sata) and target(md127 on /data1) to see how it goes.
> > > > 
> > > > If you say how you use it, someone might be able to answer
> > > > that.
> > > > Give
> > > > the exact command line with all options (e.g. if you have
> > > > strict
> > > > change
> > > > detection you might be checksumming every file on the source
> > > > before
> > > > deciding to copy it).
> > > 
> > > First, I already did this test and it still chokes after a few
> > > GB.
> > > The problem is still there
> > > and it is not only an rsync problem. But why does rsync use 100%
> > > CPU?
> > > 
> > > I thought that I gave the command but I did not initially, yet
> > > when I
> > > show the output of 'ps' it says:
> > >          sudo rsync -aHSK --stats --progress --checksum-
> > > choice=none --
> > > no-compress -W /sata/backups/tapes /data1/no-backup/old-backups/
> > > so there it is.
> > 
> > Nothing obvious at first glance. The '-W' option should mean that
> > rsync
> > is simply copying files without it's usual delta detection, which
> > could
> > have been the culprit.
> 

The culprit of rsync being slow.

> I lost you here. 'the culprit' of what? It was very slow initially.
> I then added '-W' hoping to speed it up but it did not help.
> 
> I also added '--checksum-choice=none --no-compress' for the same
> reason (probably
> not needed) but none had an effect.
> 

That's what I assumed.

> I suspect this may be related to another thread I started earlier
> 'kworker consumes 100% CPU on degraded RAID6'

Possibly. A good test would be to repeat the experiment but with a
different target destination, such as /dev/null. This would indicate
whether the problem is with reading or with writing.

poc
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue



[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux