ASort Problems

[ArneOrtlinghaus]

The Vulcan ASort function with a codeblock does not work always as the VO ASort function.
When making tests I saw that the reason seems to be strange string comparisons in VO and XS I never realized before:
"" <= "a" returns true, "a" <= "" returns also true
"" > "a" returns false, "a" > "" returns also false

These comparisons seem to be reasonable:
"" < "a" returns true, "a" < "" returns false
"" >= "a" returns false, "a" >= "" returns true
"" == "a" returns false, "a" == "" returns false

Now I ask me:
What seems to be the sense of having rules like this?
How could the VO ASort algorithm be implemented that it sorts arrays with strings correctly? I seem to understand that the Vulcan implementation cannot not do this correctly having these comparison results.
Is this influenced by the "Compatible string comparisons" or is this switch regarding the = comparison?

Arne

[Chris]

Hi Arne,

Yes, unfortunately the behavior of VO is extremely strange in this area, you will find even more strange results in VO if you make a few more tests also with "!=" operator, compare the results if you use STRING datataype holding the values with using USUALs (yes, in many cases this causes the behavior to be very different!) etc. I don't think there was a reason for having rules like that, this inconsistent behavior was definitely a bug in the original implementation of the VO and unfortunately it was carried along forever.

I remember it was a very hard job implementing all this in the vulcan runtime, but that was necessary in order for app ported from VO to have the exact same behavior as they did before. Yes, some of this VO-compatible functionality is enabled only when the /vo16 - "Compatible string comparisons" compiler option is enabled, but when it is, the behavior should be identical with VO, in all areas, including the ASort() function. If you see results that are different from VO, can you please send me or post a code sample that shows it?

Chris

[robert]

Arne,

String comparisons are a mess in VO, especially with SetExact(FALSE) which is the default setting.
It was one of the more complicated things to implement.
I will document it here (and will add this to the docs as well):

The "Compatible String Comparison (vo13)" compiler option in the compiler changes only one thing:
- Comparisons (>, >=, < and <= ) between strings will be send to a runtime function in the Vulcan Runtime DLL(__StringCompare). When this option is not set then the normal String.Compare from the .Net framework is used.
The __StringCompare runtime function takes into account the SetExact settings, and also the SetCollation() setting. When SetExact() == FALSE then it will use the .Net String.Compare and only compare the # of characters of the RHS of the expression.
When SetExact() is true then it will convert the strings from Unicode to Ansi and depending on the SetCollation flag it will either use the string table from the nation module (SetCollation(#Clipper)) or it will use the Ansi String comparison from Windows for the current Locale.
This may sound strange, but is/was needed to make sure that index files remain compatible with VO.

Regardless of the setting of this option, the "=" operator will be compiled into a call to a runtime function as well . There are a few different scenarios:

- Left Hand side (LHS) is a string and Right Hand Side (RHS) => __StringEquals()
- LHS String and RHS Usual -> __StringEquals()
- LHS Usual or RHS Usual -> __Usual.__InexactEquals()
- There are 2 overloads for __Usual.__InexactEquals()
__InexactEquals(Usual , string )
__InexactEquals(Usual , Usual )

They both link to __StringEquals() when the contents of the usuals are strings.
__StringEquals() takes into account the SetExact() setting. When SetExact() is false then it compares the # of characters of the RHS of the comparison.

Also when 2 Usuals are compared with != then the function __Usual.__InexactNotEquals() is called. Like the __InexactEquals() this function has 2 overloads. When comparing strings both of these call __StringNotEquals().
__StringNotEquals() takes into account the SetExact setting. Otherwise it will compare only the # of characters from the RHS of the comparison.

You would expact that a != operation of strings would also call __StringNotEquas but that is not the case. So != always compares as if SetExact() is TRUE.
Therefore the following produces different code

Code: Select all

FUNCTION Start AS VOID
	LOCAL cLHS AS STRING
	LOCAL cRHS AS STRING
	LOCAL uLHS AS USUAL
	LOCAL uRHS AS USUAL
	
	uLHS := cLHS := "ABCDE"
	uRHS := cRHS := "ABC"
	SetExact(FALSE)
	? 1, cLHS = cRHS      // true   
	? 2, uLHS = uRHS      // true
	? 3, cLHS = uRHS      // true
	? 4, uLHS = cRHS      // true    
	
	? 5, cLHS != cRHS      // true !
	? 6, uLHS != uRHS      // false
	? 7, cLHS != uRHS      // false
	? 8, uLHS != cRHS      // false
	
	? 9, !(cLHS = cRHS)      // false
	? 10,!(uLHS = uRHS)      // false
	? 11,!(cLHS = uRHS)      // false
	? 12,!(uLHS = cRHS)      // false
	
	RETURN
	

Number 5 is the strange one where no runtime function is called.
You will see that X# returns the same result as VO here.
Vulcan has a few errors: # 3 returns FALSE and #11 returns TRUE in Vulcan.


Robert

[ArneOrtlinghaus]

Hi Robert and Chris,
it is nice how long responses I have received. You must have spent quite a lot of time only writing the email without even thinking on the problem.

Chris has written another email to me where he tells me that during calling the VO Asort the SetExact seems to be modified.

And it is this that seems to make possible a simple workaround: Substituting the Vulcan ASort by a new ASortSafe that sets SetExact(true), calls the original ASort and resets SetExact.
At least my test cases I tried now showed the same behavior in X# as in VO. Better than having to change hundreds of old and ugly codeblocks...

Thank you for your help
Arne

[Chris]

Hi Arne,

Sorry, I probably wasn't clear, but what I meant is that ASort() in VO (at least in some of the recent versions) first compares elements for equality by using the current rules for comparison and only if this comparison results to inequality, then the codeblock you provide gets evaluated in order to decide if the 2 array elements in question must change position in the array or not. Not absolutely sure, but IIRC the current SetExact() setting is taken into account in this first internal comparison, which could lead to values like "1" and "" to be treated as equal in advance and not get passed through your codeblock for sorting.

In the vulcan runtime we forgot to adjust the implementation of ASort() to completely match this behavior, when we added general support for full VO compatible string comparisons, so right now you are right that this function's behavior is different in some cases compared to VO. We will take care of this in the x# runtime, but for now since you have implemented your own ASort() that works as expected, then I think that's the best quick solution.

Chris