On 07/01/2024 18:51, Segher Boessenkool wrote:
On Sat, Jan 06, 2024 at 06:02:45PM +0100, David Brown wrote:
On 05/01/2024 19:19, Segher Boessenkool wrote:
That's not the point. A program can be perfectly fine, with bounded
errors and all, and then -ffast-math will typically completely destroy
all that, and replace all arithmetic by the equivalent of a dice roll.
The only difference between IEEE calculations and -ffast-math
calculations is that with IEEE, the ordering and rounding is controlled
and consistent.
No, that is not the only difference.
'-ffast-math'
Sets the options '-fno-math-errno', '-funsafe-math-optimizations',
'-ffinite-math-only', '-fno-rounding-math', '-fno-signaling-nans',
'-fcx-limited-range' and '-fexcess-precision=fast'.
Many of those do much more than what you say, can result in the compiler
generating completely different code.
I know what these do - they are described in the gcc manual. And they
are all good things for the kind of code I write. But I did not list
them in my posts because it would take too much space to include them
all, every time - I have just concentrated on a couple of points.
For any given /single/ arithmetic operation that is
performed, each can have the same amount of rounding error or error due
to the limited length of the mantissa. Agreed?
I don't understand what you mean to say even.
I mean that if you write "x = a + b;" for floating point types, you
will, in general, get a rounding error. And the magnitude of the
worst-case rounding error will be the same whether you are using IEEE
rules or "-ffast-math" rules. With IEEE the rounding error will be
consistent and predictable, and for some cases that is important - but
it will not be less of a rounding error.
The rounding errors in -ffast-math will be very similar to those in IEEE
mode, for normal numbers.
No, not at all. Look at what -fassociative-math does, for example.
This can **and does** cause the loss of **all** bits of precision in
certain programs. This is not theoretical. This is real.
a = 1e120;
b = 2;
x = (a + b) - a;
IEEE rules will give "x" equal to 1e120 - mathematically /completely/
wrong. -ffast-math will give "x" equal to 2, which is mathematically
precisely correct.
The IEEE result is 0.
Sorry, of course that is what the IEEE rules will give you. It does not
help if I make silly mistakes like that!
Which is the **exactly correct** result.
It is the exactly correct result for IEEE floating point. But 2 is the
exactly correct result for modelling real number arithmetic. And for my
own use - and I believe for the majority of cases when people use
floating point - the aim of floating point in code is to model real
number arithmetic as closely as practically possible in an efficient manner.
Of course it is important, whether you use -ffast-math or not, to use
appropriate numbers and appropriate calculations - trying to evaluate
"(1e120 + 2) - 1e120" is never going to be a good idea.
But the fact remains that - for the value of "right" and "wrong" that
matters to most people - the IEEE rules will silently give you the wrong
answer here. The -ffast-math rules might give the right answer, and
might give the wrong answer. Occasionally "guaranteed wrong" is better
than "sometimes wrong" - it can sometimes make debugging and regression
testing easier. Most of the time, they are simply both bad.
Can you give an example where -fassociative-math will, as you claim,
give a result that losses /all/ bits of precision - while IEEE rules
would give a precise answer? It does not have to be all bits - I'm
happy with simply losing noticeably more bits with "-fassociative-math"
than with IEEE rules. But I want it to use the important metric for
correctness - closeness to the result using infinite precision real
arithmetic - not just closeness to the artificial value required by IEEE
rules. And I want it to be the result of realistic calculations with
realistic numbers, using a small number of calculations.
(Again, I appreciate that for some uses, predictable and consistent
results are vital even if they do not match the real arithmetic, and
IEEE rules are of great importance. I am not arguing that IEEE rules
are bad - I am arguing that -ffast-math rules are good for some uses.)
This is
a computer program, not some formulas that you can manipulate at will.
I expect the compiler to manipulate things according to its rules. The
gcc manual says how it can manipulate the floating point code I write if
-ffast-math is enabled. If I have written floating point code where the
results are equally good - by my requirements - when these manipulations
are done, then -ffast-math is safe for me and gives me correct results.
If, as you earlier suggested (exaggerating to try to make a point), the
compiler could manipulate the code to simply return 0, then I would
agree with you that the flag is dangerous and worse than useless. But
fortunately, that is not the case.
The -ffast-math flag can only reasonably be used with programs that did
not want any specific results anyway. It would be even faster (and just
as correct!) to always return 0.
That is simply wrong.
It is an exaggeration for dramatic effect, but it is fundamentally
correct.
You have vastly more knowledge than I about the internals of gcc and how
it works. You also know vastly more about IEEE floating point rules
than I do. And I expect you have worked on far more programs for which
IEEE rules are important, because they have almost never been relevant
to work I have done.
But I have regularly used floating point maths in code, in real-life
programs. I have regularly used -ffast-math, and seen how it makes my
programs faster - sometimes a great deal faster. I have read the
details of the -ffast-math flags in the gcc manual, and I know what they
do and what manipulations of my code they allow. I know what I have to
be aware of for my floating point code to give useful answers (i.e.,
answers that are at least as close to the real mathematical answers as
they need to be for the way the results are used). I know that sticking
to IEEE rules would /never/ give more useful answers for my needs - they
would give equally useful answers, slower.
Maybe there are some people who write floating point code where they get
useful answers with IEEE rules, but enabling -ffast-math would result in
useless results. I can't answer for other people - only for my own code.
And I know that what you write about -ffast-math being as useless as
"return 0" is not merely an exaggeration, it is pure FUD. And writing
it detracts from the very real factors that are always important when
writing floating point code, whether IEEE or -ffast-math, and it means
you miss out on an opportunity to discuss where the real-world
differences lie between the -ffast-math flags and full IEEE compatibility.
David
