Rounding issue

Public support forum for peer to peer support with related to the Visual Objects and Vulcan.NET products
Anonymous

Rounding issue

Post by Anonymous »

This code just will not round properly.

method EditFocusChange(oEditFocusChangeEvent) class InvoicingScr
local oControl as Control
local lGotFocus as logic
Local nGSTInc, nGSTPaid
oControl := IIf(oEditFocusChangeEvent == NULL_OBJECT, NULL_OBJECT, oEditFocusChangeEvent:Control)
lGotFocus := IIf(oEditFocusChangeEvent == NULL_OBJECT, FALSE, oEditFocusChangeEvent:GotFocus)
super:EditFocusChange(oEditFocusChangeEvent)
//Put your changes here
nGSTPaid := gGSTRate/100
nGSTInc := 1 + nGSTPaid
SetDecimal(2)
SetFixed(true)


IF AllTrim(self:oDCComboBoxEx11:CurrentItem) == "No GST"
self:oDCTOTPR1:CurrentText := Str(Round((((self:oDCQTY1:VALUE) * (self:oDCUNITPR1:VALUE)) *1),2))
self:oDCGST1:CurrentText := "0.00"
else
self:oDCTOTPR1:CurrentText := Str(Round(( ((self:oDCQTY1:VALUE) * (self:oDCUNITPR1:VALUE)) * nGSTInc),2),,2)
self:oDCGST1:CurrentText := Str(Round((((self:oDCQTY1:VALUE) * (self:oDCUNITPR1:VALUE))* nGSTPaid), 2),,2)
endif

? Round(((self:oDCQTY1:VALUE * self:oDCUNITPR1:VALUE) * nGSTInc),2)
? Str(Round(((self:oDCQTY1:VALUE * self:oDCUNITPR1:VALUE) * nGSTInc),2),,2)


SetFixed(.f.)
Return nil


Using the following variables for example:

GSTRate = 10
self:oDCQTY1:VALUE = 1
self:oDCUNITPR1:VALUE = 804.55

I keep getting 885.01 I need this to be 885.00 what am I missing please. The screenshot shows the Terminal Output for the code above:
Rounding.JPG
Rounding.JPG (28.19 KiB) Viewed 348 times
Thanks again.
.
User avatar
wriedmann
Posts: 3655
Joined: Mon Nov 02, 2015 5:07 pm
Location: Italy

Rounding issue

Post by wriedmann »

Hi Jeff,
my calculator gives me the following result:
804.55 * 1.1 = 885,01
The Windows calculator instead returns
804.55 * 1.1 = 885.005
VO is ok - but maybe you have other rules for tax calculations?
Wolfgang
P.S. why you don't type your variables nGSTInc, nGSTPaid?
Wolfgang Riedmann
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
BiggyRat

Rounding issue

Post by BiggyRat »

Hi Wolfgang, I'm not disputing the correctness of the calculation the problem is to get 885.01 to ROUND to 885.00
User avatar
wriedmann
Posts: 3655
Joined: Mon Nov 02, 2015 5:07 pm
Location: Italy

Rounding issue

Post by wriedmann »

Hi Jeff,
ok, another question:
885.005 should round to 885.00?
885.006 should round to 885.01?
Then it is easy:

Code: Select all

function JeffRound( nValue as float ) as float
local nReturn as float
    
nReturn := Round( nValue - 0.001, 2 )
	
return nReturn
Wolfgang
Wolfgang Riedmann
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
BiggyRat

Rounding issue

Post by BiggyRat »

Hi Wolfgang, no 885.01, 885.02, 885.03, 885.04 should all round DOWN to 885.00.

885.05, 885.06, 885.07, 885.08, 885.09 should round UP to 885.10

I'm afraid I tried your JeffRound function, and it gave the same result - 885.01
User avatar
wriedmann
Posts: 3655
Joined: Mon Nov 02, 2015 5:07 pm
Location: Italy

Rounding issue

Post by wriedmann »

Hi Jeff,
then you need to round to 1 decimal place and subtract the correction factor:

Code: Select all

Round( nValue - 0.01, 1 )
Wolfgang
Wolfgang Riedmann
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
BiggyRat

Rounding issue

Post by BiggyRat »

But what if the result in another transaction is 523.03? It should be 523.00, but If I understand your code correctly, it would give 523.02 wouldn't it?
User avatar
wriedmann
Posts: 3655
Joined: Mon Nov 02, 2015 5:07 pm
Location: Italy

Rounding issue

Post by wriedmann »

I'm really, really, really sorry!
If you don't understand my code, at least try it!
Wolfgang Riedmann
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
BiggyRat

Rounding issue

Post by BiggyRat »

My apologies Wolfgang, I meant no disrespect, of course I was going to try it, I was just thinking out loud. I don't get the 0.01 bit, as I thought the whole idea of the Round function was to do exactly what it said it would. I didn't think it required an undocumented "fudge factor" in order to make it work, WHICH YOUR CODE DID, So, thank you very very much again Wolfgang.
Karl-Heinz
Posts: 774
Joined: Wed May 17, 2017 8:50 am
Location: Germany

Rounding issue

Post by Karl-Heinz »

Hi Jeff,

>> I keep getting 885.01 I need this to be 885.00 what am I missing please.

you round to *two* decimal places,

>> ? Round(((self:oDCQTY1:VALUE * self:oDCUNITPR1:VALUE) * nGSTInc),2)

but it must be one decimal place to get the desired results. Run this and you see the differences.

Code: Select all

LOCAL fValue AS FLOAT 
LOCAL i AS DWORD  

fValue := 885.00


FOR i := 1 UPTO 9
	
	fValue += 0.01
	
	? Round ( fValue, 2 )  
	? Round ( fValue, 1 )  
	?	
NEXT 	 
regards
Karl-Heinz
Post Reply