Re: 2GB max file size

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Luca Ferrari wrote:

> > tar will handle input files greater than 2GB.
> >
> > I use files greater than 2GB on a RedHat 8 system with
> > a generic 2.4.20 kernel and tar (version = tar (GNU tar) 1.13.25)
> >
> > I can also use dump/restore for backing up to tape.
> 
> Actually I'm trying with zip, that fails if the file becomes bigger than 2 GB. 
> SInce I remember that, due to the i-node structure, unix cannot handle a file 
> greater than 2GB, I was wondering the problem was of the filesystem and not 
> of the program itself. However, before renouncing to use zip, is there 
> something I can do on the filesystem to handle bigger files?

The issue almost certainly lies with the program, not the filesystem.
The 2Gb limitation was lifted for ext2 so long ago that it's a distant
memory. Certainly, 2.4.20 has no problems with files larger than 2Gb.

The main problem is that the historical Unix API used a "long" to
represent offsets within files (including the offset to the end of the
file, i.e. its size). On a 32-bit system, a "long" is only 32 bits, so
can only hold values up to +/- 2Gb.

A later revision of the API introduced the off_t type to hold file
offsets (and sizes). Because of the amount of historical code which
uses the "long" type, the off_t type is equivalent to "long" by
default. This can be changed to a 64-bit type at compile-time, but you
need to ensure that all code which references the file (including any
libraries) support large files (which is why it isn't the default).

The most likely reason why you might have problems is that the zip
program was compiled to use a 32-bit off_t type. If the other programs
from your distribution support large files, that may indicate that the
zip program itself would need code modifications (and not just
compilation options) before it will support large files. Early
versions of the zip file format cannot store large files because they
only use 32 bits for the size field.

In any case, if you're backing up to tape, you should be using tar,
cpio or dump rather than zip. The zip format was designed for random
access devices (i.e. disks), not sequential access devices (i.e. 
tapes), whereas tar, cpio and dump were designed for tapes.

To create a zip archive on tape, you first have to create the file on
disk then copy it to tape; similarly, to read a zip archive, you have
to copy it to disk first. With tar, cpio or dump, you can read/write
directly from/to tape.

-- 
Glynn Clements <glynn@xxxxxxxxxxxxxxxxxx>
-
: send the line "unsubscribe linux-admin" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Newbie]     [Audio]     [Hams]     [Kernel Newbies]     [Util Linux NG]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux Device Drivers]     [Samba]     [Video 4 Linux]     [Git]     [Fedora Users]

  Powered by Linux