More info needed for xSharp on different Null(s), Nil, DotNet DBNull, None

[mainhatten]

Hi,

trying to get a clear picture on the different NULLs in xSharp.
On Bandol 2.2 there is a list of "Null Literals", which at least is missing an entry for NULL_DATETIME and perhaps for NULL_CURRENCY - second OTOH might be handled via nullable 8-bit int behind currency.

Topic of NIL is sometimes mentioned, but has no own entry beyond "NILs, assigning default values" which makes no sense to me, as it is unclear if "Assign a default value to a NIL argument" is meant for Core or VO dialect, as vfp casts missing parameters to false booleans.

Dotnet has (besides none member in various classes) its own speciality: the DBNull different from Null used on vars, see for instance
https://www.c-sharpcorner.com/blogs/han ... n-c-sharp1
and DBNull is not mentioned in xSharp docs on 2.2

Vfp memory uses "handles" as possible data type, but these handles either point to an object or other data type (including option to assign NULL value / NULL value assigned if Form is released), OTOH nullable vfp data columns can be updated/replaced with such NULL values - at the moment I see nothing mirroring the null / DBNull split of Dotnet.

Can you shed some light on how this works in xSharp ? Is "memory null" automatically transferred into DBNull and vice versa ?
Found in xSharpRDD.../DbfFPt.Prg

Code: Select all

                CASE FlexArrayTypes.NIL
                    element := DBNull.Value 

Found in xSharpCore.../Error.Prg in STATIC METHOD TypeToUsualType(oType AS System.Type) AS DWORD

Code: Select all

    CASE TypeCode.DBNull
      RETURN __UsualType.Object 

Found in xSharp.RT/.../Usual_Reftypes.Prg

Code: Select all

                    var typeCode := System.Type.GetTypeCode(vartype)
                    switch typeCode
                        case  System.TypeCode.DBNull
                            self:_flags:usualType := UsualType.Void
                            self:_refData	:= null

Also found in VOSDK/.../SQL_Functions.Prg an assign to NIL and more mentions in other Select?.Prg there as part of member names.

Cannot say I have a clear mental picture

Why ask now ? Because Robert hinted at public 2.4a release soon. All those NULL errors in String and possibly other RT functions should be doable after hammering out a schema with dev team on how and where (Vfp only? Or inside Core level?) to fix by coders outside dev team and work can be done on a current release without worry of missing something already changed in in-between FOX-only versions.

Might be better if such issues can be summed up as in "known, worked on already" as a single issue entry if new vfp coders can be enticed to test as more vfp functionality is implemented - adding a few dozen entries basically amounting to NULL-handling differences would only fog the issue and inflate issue count beyond reasonable, creating a wrong opinion on the hurdles to take till usable. Vfp IsNull() and NVL() functions could probably handled en passant.

my 0.22€ (might take a few cents worth of fixing if nobody steps up)
Thomas

[robert]

Thomas,
The concept of .NULL. like VFP has is new to people coming from VO.
On the other hand, the concept of NIL, which is common to VO people (as well as XBase++ people) is new to VFP developers.
And unfortunately they are different.
For VO developers it is clear that NIL is the value of uninitialized untyped variables (usuals).
Technically this means the usual structure is filled with all zeros.
On the other hand, for us, coming from the VO world, it was very weird to see that in VFP uninitialized variables have a value of FALSE.
We remembered that this was the case in DBase with PRIVATE and PUBLIC memory variables, but certainly not with local variables
or fields and properties.

And of course in the .Net world we now also have the concept of DbNull.Value , which seems to map the NULL value from the database world.

I think we need to come with a plan how to implement all of these in a away that works for everybody.

I have already tried to merge the concept of the VO compatible NIL and "Foxpro unitialized FALSE' in the USUAL type: when the dialect is FoxPro then the default NIL is a FALSE with a "initialized flag" set to FALSE and the type flag set to LOGIC. For other dialects (except Core, there is no USUAL in core) NIL is represented as the absense of a value, the "initialized flag" is also FALSE and the type is set to VOID (0).
The internal IsNil property of the usual type and the IsNil() runtime function now takes care of handling the different situations. So it returns TRUE when the type flag is VOID, when the Initialized flag is FALSE and also (this is a VO inconsistency) when the type is a reference type and value is NULL or when the type is PTR and the value is IntPtr.Zero.
See https://github.com/X-Sharp/XSharpPublic ... l.prg#L455

I think we somehow also need to add support for DbNull.Value in there (maybe as an additional __UsualType)

W.r.t. NULL_DATE. This is supported by the compiler but maybe not included in the help. I'll look into that
I personally see no need for NULL_CURRENCY, NULL_LONG or NULL_FLOAT. These can be either 0 or DbNull.Value.




Robert

[wriedmann]

Hi Thomas,
from what I know:

• NIL is an own datatype, and AFAIK ist can be assigned only to variables defined as usual. In the function and methods woth the Clipper calling convention not passed variables are seen as NIL
• NULL is specific to the .NET Framework (in VO we don't have it) and means AFAIK that the variable was not inizialized
• DBNull AFAIK is returned only by SQL calls

In VO, we have a null datatype for some datatypes (and VFP has the same, I think): NULL_STRING, NULL_DATE. Numerics and logic don't have a null value, they are either 0 (numeric) or false (logic).
Strings have a different behavior in .NET and in VO: a not initialized string on .NET is NULL, whereas in VO it is a NULL_STRING (not the same as ""). A NULL_STRING can be used in string comparisions, a NULL string value not. Therefore X# has a compiler switch to initialize all string variables.
Wolfgang

[mainhatten]

Hi Robert,
now just 2 items in your post raising flag here:

On the other hand, for us, coming from the VO world, it was very weird to see that in VFP uninitialized variables have a value of FALSE.
We remembered that this was the case in DBase with PRIVATE and PUBLIC memory variables, but certainly not with local variables
or fields and properties.

Nope, only PUBLIC global var is created at top level by the PUBLIC statement as boolean false. From vfp help:
PRIVATE doesn't create variables; it simply hides variables declared in higher-level programs from the current program.
Although I cannot think of any normal vfp program depending on having "uninitialzed" private as a feature. But running xSharp code under vfp might run into problems if private created and initialized variables.

W.r.t. NULL_DATE. This is supported by the compiler but maybe not included in the help. I'll look into that
I personally see no need for NULL_CURRENCY, NULL_LONG or NULL_FLOAT. These can be either 0 or DbNull.Value.

NULL_DATE is mentioned in help: NULL_DATETIME is not, unless NULL_DATE works for datetime variables as well - then the naming is not really self-explaining.

More later...

thomas

[robert]

Thomas,

Maybe Private does not create any variables, but LOCAL does. And were surprised to see that also LOCAL variables in FoxPro are initialized with FALSE. And based on all the behavior we see (locals being visible to functions like Type(), Evaluate() and (when embedded in a sql statement) SqlExec() and SqlPrepare() , they are not even true locals. They are also visible (at least to the VFP runtime) outside of the scope where they are defined.
And w.r.t. NULL_DATETIME: the datetime type does not support something like that. The closes to that are either Nullable<DateTime> or DateTime.MinValue.
The Date type however does have a NULL_DATE value, where year, month and day are all 0.

Robert

[atlopes]

I am trying to contribute to the info gathering, related only to VFP .NULL., while keeping it as neutral as possible.

Yes, all PUBLIC or LOCAL unassigned variables are initialized to .F., and the same happens to function arguments: that is, a declared parameter that is not passed to a function has the value of .F.

A variable or a property has always a type, even if it has been nullified. By default, it's Logical. Otherwise, it's of the type of the last non-null assignment (including the Object type, and therefore even if the object is gone).

Expressions can be of type .NULL., if they evaluate to .NULL.

From the two rules above:

Code: Select all

LOCAL x

y = .NULL.
? TYPE("y")  && prints L
? VARTYPE(y)  && prints X

? TYPE("x")  && prints L
? VARTYPE(x)  && prints L

Date, Datetime, and Blob variables and fields can be assigned as empty, but also can also become .NULL. If they are empty, they are not .NULL., if they are .NULL., they are not empty.

Empty literals for Date, Datetime, and Blob

Code: Select all

? VARTYPE({})   && prints D
? VARTYPE({/:})   && prints T
? VARTYPE(0h)   && prints Q

As I mentioned in another thread, when .NULL. is allowed as an argument to a function, and with some obvious exceptions like testing for .NULL., the function normally returns .NULL. (Thomas pointed out that is not the case with some string functions, but I confess I'm failing to locate which ones).

[mainhatten]

Hi Antonio

I am trying to contribute to the info gathering, related only to VFP .NULL., while keeping it as neutral as possible.

Was not aware that the thread might be interpreted as "having sides" - if I wrote something to evoke that, was not intended.

Yes, all PUBLIC or LOCAL unassigned variables are initialized to .F., and the same happens to function arguments: that is, a declared parameter that is not passed to a function has the value of .F.
.......

Code: Select all

? VARTYPE({/:})   && prints T
? VARTYPE(0h)   && prints Q

All true - but there is also IsBlank(), Append blank, Blank: those were used in 2.x times sometimes as a poor mans .Null. on Floats, Decimal, Boolean, making code brittle IMO and I hated such practice:
Perhaps this is nearest thing to the "uninitialized" concept of NIL in VO and xBase which Robert described. I only knew NIL from Pascal and Modula2 as Zeropointer or "Not in List" of a Pointer variable whose pointed to structure had been not initialised or released - therefore my hint to form.Release().
As Integer, Double and Currency don't support the "blank" concept I prefer them a lot in vfp...

As I mentioned in another thread, when .NULL. is allowed as an argument to a function, and with some obvious exceptions like testing for .NULL., the function normally returns .NULL. (Thomas pointed out that is not the case with some string functions, but I confess I'm failing to locate which ones).

I picked the ones to implement in 2.3 not only because I used them a lot, but also because I did not have to code for IsNull() changing the return values..

Code: Select all

? GETWORDCOUNT("Hallo vfp Null" + CHR(10) + "check")
? GETWORDCOUNT("Hallo vfp Null" + CHR(10) + "check", .null.)
? GETWORDCOUNT(.null.)
? GETWORDNum("Hallo vfp Null" + CHR(10) + "check",3)
? GETWORDNum("Hallo vfp Null" + CHR(10) + "check",.null.)
? ADDBS(.null.)
? FORCEEXT(.null., "txt")

and so on - comes probably from FoxTools heritage
Wanted to some other things like StrExtract(), but there you have the .Null. check needed - which could probably be done for strings in Core with "is Null" or String.IsNullOrEmpty() but instead of tying in such concepts for Usuals as well I decided to skip

regards
thomas

[atlopes]

Thomas

Was not aware that the thread might be interpreted as "having sides" - if I wrote something to evoke that, was not intended.

No, no, I was not and I am not thinking you wrote anything of the kind. I was just stressing that my contribution will not convey any judgment whatsoever regarding how VFP deals with Nulls, be it positive or negative. I'm just trying to pass info that I am aware of, as straightforward as I can.

Regarding the examples you gave, I still maintain what I said previously: when .NULL. is allowed as an argument of a VFP function, the function returns .NULL. The cases you presented do not allow .NULL. as parameters, and they raise an error, instead.

The nullification also affects expressions in which .NULL. values are used as operands. More examples:

Code: Select all

? SQRT(.NULL.)   && prints .NULL.
? DATETIME() + .NULL.   && prints .NULL.
? 25 / .NULL.   && prints .NULL.

Exceptions that I know of: logical AND operations in which one of the operands is .F., or logical OR operations in which one of the operands is .T.

[mainhatten]

Antonio

Regarding the examples you gave, I still maintain what I said previously: when .NULL. is allowed as an argument of a VFP function, the function returns .NULL. The cases you presented do not allow .NULL. as parameters, and they raise an error, instead.

In my diction anything not forbidden/constrained in the docs (for example describing allowed parameter types or count) is allowed. For instance often possible data types are described.
The return of .null. from strextract() when called with a .null. parameter makes as much sense to me as returning .null. from addBS() would - not a really usable return value, so IMO throwing an error there could be exchanged for returning .null. without much difference to calling program when looked at it from eagle eye position - would need other error checking/wrapping, but result still unusable. Or - put in your "Nullification" context - when

Code: Select all

lxNull .= Null + ""
? lxNull

resolves to null, why stop AddBS() from doing the same, as docs does not mention .null. values there or at StrExtract? I classify error thrown in AddBS() as implementation detail same as returning .null. is in StrExtract().

I had wondered a bit on your choice of words with "allowed", as Vfp does not support the distinction of nullable and not nullable types which were part of Kotlins lure over Java as integral part of the language and which Dotnet introduced early on as well, while much of Java depends on annotations (IMO too much, Lombok already at the border of "too much magic").

Interpreted it as encompassing statement including languages like Kotlin/C#, whereas you intended to mark "throwing errors" as "not allowed". I still think "errorthrowing" in those functions comes from directly compiling C++ code written for FPD/FPW, where there was no concept of .null. inside the language and the handling of functions like StrExtract() more in line of functions usable from SQL, where an error thrown on 1 particular record with a .null. value in worked on column is worse than returning .null. for that particular row. Result is still "I don't know", but processing can continue if table allowed nullable column value. Might not be that clear in AddBS() case, but JustExt() can be used on columns as well as "pattern analysis", but carries that threat making direct SQL usage brittle.

my 0.02€
thomas

[atlopes]

Thomas,

I don't want to be too picky on this matter. As I said, I aim to help gather as much information as possible to the X# team so that they can make their decisions on the VFP implementation.

That said, and starting to get a bit judgmental here, I don't think that VFP's NULL approach is consistent all the way in every corner, I'm fully agreeing with you. It won't be too hard to find some Wat moments.

Furthermore, in their specific area, VFP SQL commands (the great mountain ahead for the X# team to climb) follow the ANSI rules. For instance,

Code: Select all

CREATE CURSOR x (n Int NULL)
INSERT INTO x VALUES (1)
INSERT INTO x VALUES (.NULL.)
SELECT MAX(n) FROM x  && shows 1

The only reasonable scenario I anticipate where we would throw a NULL inside a built-in VFP function is when the value comes from an external source, like a query, or a result from a method call, or the value of a property. But this is me not being neutral anymore.

Nevertheless, I think that being able to distinguish between NULL and any other value, including empty values, should be a fairly expectable target for a VFP implementation, and I hope X# succeeds in this.