both operands are checked for infinity first.
Not really. The code reads /* Handle infinities */ if (IS_INF(dest)) { if (IS_ZERO(src)) fp_set_nan(dest); return dest; } So the src argument is checked for ZERO first. Similar in the implementation of fgetman: if (IS_ZERO(dest)) return dest; if (IS_INF(dest)) return dest;
I cannot locate the specific assembly function you mention (fp_long_ext2ext). Can you post code or disassembly to illustrate the problem?
A sorry, i meant fp_conv_ext2ext. It is called by the fp_normalize_ext() macro in fp_emu.h, which in turn is called by fp_monadic_check/fp_dyadic_check.
That might be a genuine error in the sqrt algorithm there - can't see how it does arise though.
I think this is caused by dest->exp += (exp - 0x3FFF) / 2; `exp` here is the original exponent of the source, and if that is < 1.0, the expression `exp - 0x3FFF` will become negative, but the division truncates it towards zero. IMHO using `>> 1` should fix that.
You may have to look elsewhere for a feature complete FPU emulation (does netbsd have that?).
Yes, NetBSD has something similar, and it seems to be almost complete, including all the trigonometric functions. However they do everything in C, including all the <ea> calculations. The linux version seems to be much better in this regard.