Re: out of memory

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Oct 06, 2010 at 08:48:24PM -0400, Paul Dugas wrote:
> I'm working with an up-to-date CENTOS-5 x86_64 machine connected via
> ATA-over-Ethernet to a Coraid array.  I have one drive in the array
> that I want to use for manually transported offsite backups and I want
> the drives to be encrypted.  I've done a fair bit of reading but am
> having trouble.  Since I can't really find a single source of the
> trouble, I thought it was time to ask for some input.
> 
> As I mentioned earlier, the drive is in an AoE array.  It's setup as a
> single drive (JBOD, no raid) and it properly appears as
> /dev/etherd/e1.12; 12th drive in shelf 1.  The drive capacity is 2TB.
> I started like so:
> 
>      # cryptsetup -y luksFormat /dev/etherd/e1.12
>          confirm and set password
>      # cryptsetup luksOpen /dev/etherd/e1.12 offsitefs
>          give it the password
>      # cryptsetup status offsitefs
>          just checking
> 
> No trouble up to here.  Then, based on guidance found online, I ran dd like so.
> 
>      # dd if=/dev/urandom of=/dev/mapper/offsitefs
>          come back tomorrow for the dd to be finished...
> 
> The first time I tried this, the machine almost immediately went to
> its knees.  Console and terminal sessions staggered to a crawl.
> Messages on the console indicated the machine was out of memory.  I
> eventually gave up on it and cycled power.
> 
> I tried it again but was prepared with some terminals running top and
> other monitors before I started the dd.  I saw the dd process
> consuming some CPU but not terribly much.  This time, the dd evenually
> completed.  It took a couple days but it did eventually complete.
> 
> Next, I setup the filesystem like so:
> 
>      # mke2fs -j -O dir_index /dev/mapper/offsitefs
>          create ext2+journal filesystem
> 
> This also brought the machine to its knees but after a couple
> power-cycle iterations and a couple more days, it completed and I was
> able to mount the drive and use it to a point.  I was able copy most
> of our backup files (tar.gz files) to the drive but the largest one
> (over 300GB) failed.  I didn't catch why as I'd let it chug along
> without me again.
> 
> My first question is whether or not I've made some fundamental error
> in the approach I'm taking.  Should this work?

It should. I don't see any obvious problem.
 
> My second question is if I just need to add more RAM to the machine
> (currently on 2GB).

Not really. LUKS does not need a lot of memory.

> Third, what additional information can I collect to narrow down the
> issues I'm having?

The question would be where the memory actually goes. Is there maybe
some bus or disk (i.e. hardware) problem that slows down writes to
the disk and the memory all goes to the buffer? Or does it go 
to some process? top or ps -auxwww + free should bring insights
into that.

One thing you can try is to use dd_rescue instead of dd, as it
gives progress statistics. Then you can try to zero the raw disk
and see whether that goes with the expected speed. This will kill
what you already have, sorry. But there seems to be something
fundamentally wrong on writing. 

BTW, you can just zero /dev/mapper/offsitefs, reading from urandom
is slow and completely unnecessary, read from /dev/zero instead.


Arno

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@xxxxxxxxxxx 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 
_______________________________________________
dm-crypt mailing list
dm-crypt@xxxxxxxx
http://www.saout.de/mailman/listinfo/dm-crypt


[Index of Archives]     [Device Mapper Devel]     [Fedora Desktop]     [ATA RAID]     [Fedora Marketing]     [Fedora Packaging]     [Fedora SELinux]     [Yosemite News]     [KDE Users]     [Fedora Tools]     [Fedora Docs]

  Powered by Linux