Eureka!!! The simplest answer is usually the right
one.
I tried a sanity check rewriting that script in a
simple C prog using ?w?, ?w+?, and ?r+? and sure
enough it looks like truncation is what I was seeing.
(Never though to check the differences concerning
truncation between the three; r+ works, btw, and
ironically it is the only one of the three that says
it doesn?t truncate, go figure.)
The obvious answer may have been in front of my nose
all the time? Thanks!!
(BTW I am validating a product and haven?t actually
seen the source code, but I?d bet 90% odds this is the
root cause.)
--- "Mermell, Todd" <mermell@xxxxxxxxxx> wrote:
> Hi there,
>
> In your example, you are using dd, which is going to
> truncate the file each time, resulting in
> reallocation of the data blocks:
>
> open("./<somefile>",
> O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 1
>
> (where <somefile> is an existing file)
>
> You might want to check what method you are using to
> overwrite the file, i.e. C, perl, shell, etc. If
> you're making the read/write or pread/pwrite system
> calls, then you might want to just check what flags
> are being passed to "open."
>
> -Todd
>
>
> -----Original Message-----
> From: hell know [mailto:sp0ck1701@xxxxxxxxx]
> Sent: Thu 10/7/2004 6:33 PM
> To: ext3-users@xxxxxxxxxx
> Cc:
> Subject: Ext 2/3 overwriting remnant data & use of
> data blocks - security
> Greetings all-
>
> I am conducting security testing on a device that
> uses Linux 2.4 with ext3. I am testing secure
> overwrite of remnant data in temporary files, but
> have run into a real good stumpper in the way Ext
> allocates data blocks. I've got 10 yrs of *NIX
> behind me, several with Linux, and this has really
> got me perplexed as I can't find any documentation
> explaining the subject enough for me to figure out
> what's going on so I'm hoping someone can help shed
> some light on this... any help would be appreciated!
>
> BACKGROUND:
> Device under test uses temporary spool files. When
> those files are no longer needed, they are to be
> overwritten by the three-pass DOD overwrite (pattern
> '35', 'ca', '97'), then deleted. (Incase anyone out
> there asks the obvious question, I am aware that Ext
> supports a "secure" attribute but unfortunately that
> isn't enough for our purposes. It HAS to be a
> 3-pass overwrite... afterall that answer would be
> TOO EASY ;-). Also, the file is written and
> overwritten sequentially- that may be important to
> know when I get to the problem.
>
> What is supposed to happen is this:
> 1) File is created. Inode allocated. Data blocks
> allocated, etc. Initial data is put into the file.
> {For example, lets say the file occupies data blocks
> 100 - 200 }
>
> 2) File is read/processed. (And is then no longer
> needed)
>
> 3) File is overwritten three times from beginning
> to end to overwrite the remnant data:
>
> 3a) with pattern '35'
> { If I go in with dd and check data blocks 100-200 I
> should see all '35' }
>
> 3b) with pattern 'ca' { data blocks 100-200 should
> be all 'ca' }
> { If I go in with dd and check data blocks 100-200 I
> should see all 'ca' }
>
>
> 3c) with pattern '97' { data blocks 100-200 should
> be all '97 }
> { If I go in with dd and check data blocks 100-200 I
> should see all '97' }
>
>
> 4) File is deleted (inode/dir entry zapped per
> typical *NIX behavior) {data blocks 100-200
> deallocated }
> { If I go in with dd and check data blocks 100-200 I
> should see all '97' still, until the blocks are
> reused by another file }
>
>
> Makes good sense, right? ;-)
>
> PROBLEM:
> I'm not seeing the behavior described above. Linux
> keeps "shifting" the data blocks around each time
> the file is written to. So using our hypothetical
> data block numbering, I see something like this
> occurring:
>
> 1) File is created. Inode allocated. Data blocks
> allocated, etc. Initial data is put into the file.
> {Data blocks 100 - 200 }
>
> 2) File is read/processed. (And is then no longer
> needed)
>
> 3) File is overwritten three times from beginning
> to end to overwrite the remnant data:
>
> 3a) with pattern '35'
> { data blocks 300-400 get written with all '35', not
> 100-200 }
> { If I go in with dd and check data blocks 100-200 I
> still see the original data }
>
>
> 3b) with pattern 'ca'
> { data blocks 500-600 get written with all 'ca', not
> 100-200 }
> { If I go in with dd and check data blocks 100-200 I
> still see the original data }
>
>
>
> 3c) with pattern '97' { data blocks 100-200 should
> be all '97 }
>
> { data blocks 700-800 get written with all '97', not
> 100-200 }
> { If I go in with dd and check data blocks 100-200 I
> still see the original data }
>
>
>
> 4) File is deleted (inode/dir entry zapped per
> typical *NIX behavior) {data blocks 100-200
> deallocated }
>
> { If I go in with dd and check data blocks 100-200 I
> still see the original data until the data blocks
> are overwritten by another file }
>
>
> So you can see my problem. I suspect it has
> something to do with the way Ext 2/3 preallocates
> blocks, and it's use of "block groups". But I have
> been unable to locate any good docs which clearly
> explain exactly what algorithm logic controls this,
> and how to either change it or work with it somehow
> for the overwrite process.
>
> Can anyone shed any light into this? Am I heading
> in the right direction here? I'm not afraid to read
> any docs if you can point me to them. (Or if it can
> be explained simply and you're willing to type it
> out, that's great too!)
>
> MORE INFO
> As proof of concept, I wrote the simple script
> below. As you can see all it does is open a file,
> write a chunk of data to it (simulating the initial
> data write), and then repeats that process 25 times
> (simulating cycles of overwrites); each time it
> grabs the list of data blocks using debugfs and its
> "stat" command. And for my immediate purposes, the
> first 12 direct blocks are sufficient-- don't want
> to even think about indirect blocks until I get this
> behavior figured out.
>
> Before I get to the tty dumps, I want to say THANK
> YOU to anyone who's read this post this far... and
> of course, thanks in advance to any help!!!
>
> The pertinent info:
> # uname -a
> Linux vrh90 2.4.20-8 #1 Thu Mar 13 17:18:24 EST 2003
> i686 athlon i386 GNU/Linux
> # mount
> /dev/hda2 on / type ext3 (rw)
> none on /proc type proc (rw)
> /dev/hda1 on /boot type ext3 (rw)
> none on /dev/pts type devpts (rw,gid=5,mode=620)
> none on /dev/shm type tmpfs (rw)
> //192.168.0.1/data1 on /mnt type smbfs (0)
>
> ----
> for (( i=0;i<20;i=i+1 )); do
> echo Pass $i
> dd if=/dev/urandom of=/tmp/target1 bs=1 count=102400
> debugfs /dev/hda2 -R "stat /tmp/target1" 2>
> /dev/null | tee -a /tmp/log
> done
> ----
>
> Here is the "log" file containing debugfs "stat
> /tmp/target1" for each cycle. (I can see a definate
> pattern on some of these in which blocks are used,
> but again I'm trying to figure out what the
> algorithm is that causes it):
>
> ----
> Inode: 209219 Type: regular Mode: 0644
> Flags:
=== message truncated ===
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
_______________________________________________
Ext3-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/ext3-users
