Float, Real8, or Decimal?

[ArneOrtlinghaus]

I have now verified what is the setfloatdelta in our programs without having specified it.
It gives:
0,0000000000001

Now I can again think about what I have written some minutes before and the consequences...

... Fortunately in our programs we should have already made more stable most of the floating comparisons using the following operations.
function NumEQU (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob zwei Werte bis auf Rundungen gleich sind
//g Math
//g General
if Abs(nWert1) < 1.E3
return (abs(nWert2-nWert1) < (1.E-11))
elseif Abs(nWert1) < 1.E6
return (abs(nWert2-nWert1) < (1.E-8))
elseif Abs(nWert1) < 1.E9
return (abs(nWert2-nWert1) < (1.E-5))
else
return (abs(nWert2-nWert1) < (1.E-3))
endif

function NumGEQ (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob ein Wert bis auf Rundungen größer gleich als ein anderer ist
//g Math
//g General
return NumEQU (nWert1,nWert2) .or. nWert1 > nWert2

function NumGTR (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob ein Wert bis auf Rundungen größer als ein anderer ist
//g Math
//g General
return !NumEQU (nWert1,nWert2) .and. nWert1 > nWert2

function NumLEQ (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob ein Wert bis auf Rundungen kleiner gleich ein anderer ist
//g Math
//g General
return NumEQU (nWert1,nWert2) .or. nWert1 < nWert2

function NumLSS (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob ein Wert bis auf Rundungen kleiner als ein anderer ist
//g Math
//g General
return !NumEQU (nWert1,nWert2) .and. nWert1 < nWert2

function NumNEQ (nWert1 as real8,nWert2 as real8) as logic strict
//p Prüfung, ob zwei Werte bis auf Rundungen ungleich sind
//g Math
//g General
return !NumEQU (nWert1,nWert2)

[Chris]

Hi Arne,

The problem is the precision of the real8 data type with decimal numbers, if you try this in any language:

LOCAL r1,r2 AS REAL8
r1 := 0.1
r2 := 0.3
r1 := r1 * 3.0
? r1 == r2 // FALSE!

this will return FALSE in all languages, including VO, X#, c# etc, because in binary format the value 0.1 cannot be exactly represented, so when doing math on it, you just go further away from the actual value.

VO attempted to overcome this inherent limitation of floating types by introducing SetFloatDelta(), which does work, but up to a point only, before needing to specify a larger value for SetFloatDelta(). In .Net this was fixed by introducing the System.Decimal data type, which can accurately represent decimal numbers, so you do not have to worry at all about the above anymore. Only problem is that it is slower than REAL8, which might become a problem if your calculations are extremely extensive.

Chris

[Otto]

Hi Chris,
I didn't even know that there is this setfloatdelta after 20 years of VO programming....

same here... oh brother... would have saved us a lot of trouble.

[wriedmann]

Hi Arne, hi Otto,

using SetFloatDelta() is a relatively easy possibility to keep the problems of the floating point mess as low as possible.

I'm very happy that such a thing like the Decimal datatype exists in .NET!

Wolfgang
P.S. my first programming experience was Cobol, and there is absolutely no problem with decimal variables if you don't use binary datatypes.

[lumberjack]

I didn't even know that there is this setfloatdelta after 20 years of VO programming....

same here... oh brother... would have saved us a lot of trouble.

It makes me think back to this in the c.l.c.vo NG...

Code: Select all

Willie Moore 	
8/31/07
Hey,
Here is a small tip and function in you ever run into any VB code that you need to translate to VO. in VB, int returns the integer part of the number ignoring the fractional part. But, if the number is negative, it take the abs of the fraction and if it is greater than 0, it returns the next negative number less than the passed number. So, if the number is int(-8.01) -n VB returns -9. (http://msdn2.microsoft.com/en-us/library/xh29swte(VS.71).aspx)
If you have to convert VO code from VB, here is a function to handle it.
Regards,
Willie

FUNCTION vbInt(nVar AS FLOAT) AS LONG PASCAL
   LOCAL nResult AS LONG
   LOCAL fFrac   AS FLOAT
   IF nVar >= 0
       nResult := INT(nVar)
   ELSE        
      fFrac := Frac(nVar)
      IF Abs(fFrac) > 0    
          nResult := INT(nVar) - 1
      ELSE
         nResult := INT(nVar)
     ENDIF
   ENDIF
   RETURN nResult

Code: Select all

Johan Nel
8/31/07
Hi Willie,
Yes you right, VB and VO handle negative integers differently.
You could also just use:
FUNCTION vbInt(nVar)
  RETURN Floor(nVar)

Code: Select all

Willie Moore 	
8/31/07
Johan,
Thanks. Dont think I have ever used floor. Makes the function ever more simple.
Regards,
Willie

[Otto]

Seems like SetFloatDelta doesn't work for Real8 in XSharp (VO).
In VO it works for both Real8 the same way as for Float.

[robert]

Otto,
Thanks for the report.
To make this work we will have to change the compiler to generate different code for Real8 comparisons, a bit like we are doing now for string comparisons (/vo13) and we will have to add comparison routines to the runtime.
This is not very difficult but will take some time. Most likely we would add a new compiler option (vo17) for this.
How important is this for you ?

Robert

[Chris]

Hi Otto,

Are you sure? I just tried this in VO and it returns FALSE for REAL8 and TRUE (as expected) for FLOAT:

Code: Select all

LOCAL r1,r2 AS REAL8
LOCAL f1,f2 AS FLOAT
SetFloatDelta(0.1)
r1 := 0.01
r2 := 0.02
f1 := r1
f2 := r2
? r1 == r2 // FALSE
? f1 == f2 // TRUE

[Otto]

I've included the VO version.
But I compare the Real8 with a literal 0.0
Could it be that the 0.0 is seen as a float and that therefore the left hand side (real8) is converted into a float as wel?

[Otto]

About the importance:
I'm looking for solutions for a lot of comparisons in my code between literal zero's and real8's.

If I can figure a structural solution for this in another way that could be ok. I am aware of the trouble in general with floating point comparison...

I'll come back on this tomorrow.