On Mon, Apr 26, 2004 at 12:48:45PM -0700, Dann Corbit wrote:mod(x, y) is computed as x - trunc(x / y) * y in the mod_var() function (I think).
Maple output:
y := 123456789012345678901234567890 mod 123;
y := 117
PgSQL 7.3.6 gives the right answer (117), 7.4 gets it wrong (-6). Most likely a bug was introduced when NUMERIC was rewritten. Strange it hasn't been noticed before.
However, it appears that the division operator itself is rounding up, such that the trunc() function (which ought to round down) does no good as a round up has already occurred.
Thus, the value of (x / y) is 1 too large, and so x % y is actually giving you (x % y) - y, a negative number. I tried looking at how the division actually works, but it is over my head at least for the 30 minute perusal.
Regards, Paul Tillotson
-----------------------------------------------------------------------
[paul@pjt4 paul]$ bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. 111111111111111111 / 6 18518518518518518
[paul@pjt4 bin]$ ./psql -U postgres template1 Welcome to psql 7.4.2, the PostgreSQL interactive terminal.
Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quittemplate1=# select 111111111111111111::numeric / 6;
?column?
-------------------
18518518518518519
(1 row)template1=# select 111111111111111111 / 6;
?column?
-------------------
18518518518518518
(1 row)template1=# select version();
version
---------------------------------------------------------------------------------------------------------
PostgreSQL 7.4.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.3.2 20031022 (Red Hat Linux 3.3.2-1)
(1 row)
---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
