Changing subject, so that there is at least a chance for someone to find
this discussions with a search engine :)
Joanne,
jdow - 28.06.18, 04:57:
The issue is what happens when one of those disks appears on a 3.1
system. {^_^}
That is right, so I think the warning about 64 bit support in native OS
is okay, but that issue already exists *without* Linux.
Remember, I created that RDB with Media Toolbox on AmigaOS 4.0. I did
not even use Linux to create that beast :). If it booms in AmigaOS < 4
without NSD64, TD64 or SCSI direct, that would happen with or without
the warning in Linux, even without the disk ever have been seen by a
Linux kernel.
I´d say the warning about support in native OS does not harm, even when
it is about educating Amiga users who, in case they use that large
drives – and I pretty much bet, some of them do –, better know the
limitations beforehand.
I do think the extra kernel option does not make all that much sense,
but I accept it anyway.
Cause if you argue like that, what would need fixing is AmigaOS < 4
without NSD64, TD64 or SCSI direct, but then that is what NSD64 and TD64
was made for more than 10 years ago.
Of course, if a partitioning tool for Linux ever allows to create such
an RDB, it makes sense to add a big fat warning about that. As… I think
would make sense to have in Media Toolbox and AmigaOS partitioning
tools.
However Linux here just reads the RDB, so I´d personally go with the
warning about support in native OS, but spare myself the extra kernel
option stuff.
It is Michael´s call tough, as he submits the patch. And if he chooses
to be on a safer side than this, that is fine with me.
Thanks,
Martin
On 20180627 01:03, Martin Steigerwald wrote:
Dear Joanne.
jdow - 27.06.18, 08:24:
You allergic to using a GPT solution? It will get away from some of
the evils that RDB has inherent in it because they are also
features?
(Loading a filesystem or DriveInit code from RDBs is just asking
for
a nearly impossible to remove malware infection.) Furthermore, any
32
bit system that sees an RDSK block is going to try to translate it.
If you add a new RDB format you are going to get bizarre and
probably
quite destructive results from the mistake. Fail safe is a rather
good notion, methinks.
Personally I figure this is all rather surreal. 2TG of junk on an
Amiga system seems utterly outlandish to me. You cited another
overflow potential. There are at least three we've identified, I
believe. Are you 100% sure there are no more? The specific one you
mention of translating RDB to Linux has a proper solution in the
RDB
reader. It should recover such overflow errors in the RDB as it can
with due care and polish. It should flag any other overflow error
it
detects within the RDBs and return an error such as to leave the
disk
unmounted or mounted read-only if you feel like messing up a poor
sod's backups. The simple solution is to read each of the variables
with the nominal RDB size and convert it to uint64_t before
calculating byte indices.
However, consider my inputs as advice from an adult who has seen
the
Amiga Elephant so to speak. I am not trying to assert any control.
Do
as you wish; but, I would plead with you to avoid ANY chance you
can
for the user to make a bonehead stupid move and lose all his
treasured disk archives. Doing otherwise is very poor form.
I am pretty confident that larger than 2 TiB disks are fully
supported within AmigaOS 4, as I outlined in my other mail.
So with all due respect: I used a larger than 2 TiB disk in AmigaOS
4 in 2012 already *just* fine. I even found I had the same
questions back then, and researched it. Which lead to this official
article back then:
http://wiki.amigaos.net/wiki/RDB
I am also pretty sure that AmigaOS still uses RDB as partitioning
format. They support MBR. I don´t think AmigaOS 4.1 supports GPT.
Whether to implement that of course is the decision of AmigaOS 4
development team. I am no longer a member of it since some time.
Linux m68k should already be able to use disks in GPT format, but
you
likely won´t be able to read them in AmigaOS, unless there is some
third party support for it meanwhile.
Thanks,
Martin
{o.o} Joanne "Said enough, she has" Dow
On 20180626 18:07, Michael Schmitz wrote:
Joanne,
As far as I have been able to test, the change is backwards
compatible (RDB partitions from an old disk 80 GB disk are still
recognized OK). That"s only been done on an emulator though.
Your advice about the dangers of using RDB disks that would have
failed the current Linux RDB parser on legacy 32 bit systems is
well
taken though. Maybe Martin can clarify that for me - was the 2 TB
disk in question ever used on a 32 bit Amiga system?
RDB disk format is meant for legacy use only, so I think we can
get
away with printing a big fat warning during boot, advising the
user
that the oversize RDB partition(s) scanned are not compatible with
legacy 32 bit AmigaOS. With the proposed fix they will work under
both AmigaOS 4.1 and Linux instead of truncating the first
overflowing partition at disk end and trashing valid partitions
that overlap, which is what Martin was after.
If that still seems too risky, we can make the default behaviour
to
bail out once a potential overflow is detected, and allow the user
to
override that through a boot parameter. I'd leave that decision up
for the code review on linux-block.
Two more comments: Linux uses 512 byte block sizes for the
partition
start and size calculations, so a change of the RDB blocksize to
reduce the block counts stored in the RDB would still result in
the
same overflow. And amiga-fdisk is indeed utterly broken and needs
updating (along with probably most legacy m68k partitioners).
Adrian
has advertised parted as replacement for the old tools - maybe
this
would make a nice test case for parted?
Cheers,
Michael
On Tue, Jun 26, 2018 at 9:45 PM, jdow <jdow@xxxxxxxxxxxxx> wrote:
If it is not backwards compatible I for one would refuse to use
it.
And if it still mattered that much to me I'd also generate a
reasonable alternative. Modifying RDBs nay not be even an
approximation of a good idea. You'd discover that as soon as an
RDB uint64_t disk is tasted by a uint32_t only system. If it is
for your personal use then you're entirely free to reject my
advice and are probably smart enough to keep it working for
yourself.
GPT is probably the right way to go. Preserve the ability to read
RDBs for legacy disks only.
{^_^}
On 20180626 01:31, Michael Schmitz wrote:
Joanne,
I think we all agree that doing 32 bit calculations on 512-byte
block
addresses that overflow on disks 2 TB and larger is a bug,
causing
the issues Martin reported. Your patch addresses that by using
the correct data type for the calculations (as do other
partition
parsers that may have to deal with large disks) and fixes
Martin's bug, so appears to be the right thing to do.
Using 64 bit data types for disks smaller than 2 TB where
calculations don't currently overflow is not expected to cause
new issues, other than enabling use of disk and partitions
larger
than 2 TB (which may have ramifications with filesystems on
these
partitions). So comptibility is preserved.
Forcing larger block sizes might be a good strategy to avoid
overflow
issues in filesystems as well, but I can't see how the block
size
stored in the RDB would enforce use of the same block size in
filesystems. We'll have to rely on the filesystem tools to get
that right, too. Linux AFFS does allow block sizes up to 4k (VFS
limitation) so this should allow partitions larger than 2 TB to
work already (but I suspect Al Viro may have found a few issues
when he looked at the AFFS code so I won't say more). Anyway
partitioning tools and filesystems are unrelated to the Linux
partition parser code which is all we aim to fix in this patch.
If you feel strongly about unknown ramifications of any
filesystems on partitions larger than 2 TB, say so and I'll have
the kernel print a warning about these partitions.
I'll get this patch tested on Martin's test case image as well
as
on a RDB image from a disk known to currently work under Linux
(thanks Geert for the losetup hint). Can't do much more without
procuring a working Amiga disk image to use with an emulator,
sorry. The Amiga I plan to use for tests is a long way away from
my home indeed.
Cheers,
Michael
Am 26.06.18 um 17:17 schrieb jdow:
As long as it preserves compatibility it should be OK, I
suppose.
Personally I'd make any partitioning tool front end gently
force
the
block size towards 8k as the disk size gets larger. The file
systems
may also run into 2TB issues that are not obvious. An unused
blocks
list will have to go beyond a uint32_t size, for example. But a
block
list (OFS for sure, don't remember for the newer AFS) uses a
tad
under 1% of the disk all by itself. A block bitmap is not quite
so bad. {^_-}
Just be sure you are aware of all the ramifications when you
make
a
change. I remember thinking about this for awhile and then
determining I REALLY did not want to think about it as my brain
was getting tied into a gordian knot.
{^_^}
On 20180625 19:23, Michael Schmitz wrote:
Joanne,
Martin's boot log (including your patch) says:
Jun 19 21:19:09 merkaba kernel: [ 7891.843284] sdb: RDSK
(512)
sdb1
(LNX^@)(res 2 spb 1) sdb2 (JXF^D)(res 2 spb 1) sdb3
(DOS^C)(res
2 spb
4)
Jun 19 21:19:09 merkaba kernel: [ 7891.844055] sd 7:0:0:0:
[sdb]
Attached SCSI disk
so it's indeed a case of self inflicted damage (RDSK (512)
means
512
byte blocks) and can be worked around by using a different
block
size.
Your memory serves right indeed - blocksize is in 512 bytes
units.
I'll still submit a patch to Jens anyway as this may bite
others
yet.
Cheers,
Michael
On Sun, Jun 24, 2018 at 11:40 PM, jdow <jdow@xxxxxxxxxxxxx>
wrote:
BTW - anybody who uses 512 byte blocks with an Amiga file
system is
a famn
dool.
If memory serves the RDBs think in blocks rather than bytes
so
it
should
work up to 2 gigablocks whatever your block size is. 512
blocks
is
2199023255552 bytes. But that wastes just a WHOLE LOT of disk
in
block maps.
Go up to 4096 or 8192. The latter is 35 TB.
{^_^}
On 20180624 02:06, Martin Steigerwald wrote:
Hi.
Michael Schmitz - 27.04.18, 04:11:
test results at
https://bugzilla.kernel.org/show_bug.cgi?id=43511
indicate the RDB parser bug is fixed by the patch given
there, so if
Martin now submits the patch, all should be well?
Ok, better be honest than having anyone waiting for it:
I do not care enough about this, in order to motivate myself
preparing the a patch from Joanne Dow´s fix.
I am not even using my Amiga boxes anymore, not even the
Sam440ep
which
I still have in my apartment.
So RDB support in Linux it remains broken for disks larger 2
TB,
unless
someone else does.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe
linux-m68k" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe
linux-m68k" in the body of a message to
majordomo@xxxxxxxxxxxxxxx
More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe
linux-m68k" in the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
Martin
--
To unsubscribe from this list: send the line "unsubscribe linux-m68k" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
