Lew wrote:
Postgres User wrote:
The problem turned out to be related to my function..
Given this table:
CREATE TABLE "table2" (
"s_val" numeric(6,2),
"e_val" numeric(6,2)
) WITH OIDS;
The following functions of code will set retval = NULL;
declare
retval numeric(6,2);
rec record;
begin
SELECT * INTO rec FROM table2 LIMIT 0;
rec.s_val = 100;
rec.e_val = 101;
retval = (rec.s_val - rec.e_val) / rec.s_val;
return retval;
end
However, if I explicitly typecast, then it returns the proper value:
retval = (rec.s_val::numeric(6,2) - rec.e_val::numeric(6,2)) /
rec.s_val::numeric(6,2);
Sure, because the first way you're doing integer division, and the
second way you're doing floating point division. In integer division,
-1/100 yields zero.
The more I look at this, the more I think I'm wrong.
I'm researching the semantics of the idioms that you used. I don't know what
type rec.s_val and rec.e_val end up being after the integer assignments.
--
Lew
---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings
