A call to PInvoke function has unbalanced the stack

This forum is meant for questions and discussions about the X# language and tools
User avatar
baramuse
Posts: 87
Joined: Tue Nov 29, 2022 8:31 am
Location: France

A call to PInvoke function has unbalanced the stack

Post by baramuse »

Hi everyone,

in our VO app, we were using a C dll for encryption/decryption
The ported code seemto crash when calling the dll.
We do have the source code of the C dll so I tried to see what was bother the Xsharp version of the code :

Code: Select all

#define BF_LONG unsigned long
#define BF_ROUNDS	16

typedef struct bf_key_st
{
	BF_LONG P[BF_ROUNDS+2];
	BF_LONG S[4*256];
} BF_KEY;

typedef BF_KEY	PCRYPT_CTX;

void PCryptInit(PCRYPT_CTX *ctx, unsigned char *key, int count)
{
	BF_set_key(ctx, count, key);
	return;
}
Now on the XSharp side here is the code that was generated by Xporter :

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

VOSTRUCT PCRYPT_CTX
    MEMBER DIM P[16+2]	AS DWORD
    MEMBER DIM S[4*256]	AS DWORD
When using that I have a exception :
A call to PInvoke function has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature.

So I tried to match the signature "closer"

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS BYTE[], count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

USING System.Runtime.InteropServices
[StructLayout(LayoutKind.Sequential)];
PUBLIC CLASS PCRYPT_CTX
    PUBLIC DIM P[16+2]	AS DWORD
    PUBLIC DIM S[4*256]	AS DWORD
END CLASS

LOCAL ctx		AS PCRYPT_CTX
LOCAL bKey		AS BYTE[]
bKey := BYTE[]{16}{235,89,56,22,81,127,158,80,216,87,210,149,134,206,99,129}
    
PCryptInit(ctx, bKey, bKey:Length)
Same exception, maybe I'm not looking in the right place ?
Any suggestion ?
User avatar
Chris
Posts: 4599
Joined: Thu Oct 08, 2015 7:48 am
Location: Greece

A call to PInvoke function has unbalanced the stack

Post by Chris »

Hi Basile,

Can you please show also the exact code you were using to call the function with its original definition?

Also please note that in the modified version of the code, you are passing the first parameter "ctx" (which represents a regular type) uninitialized, so this will definitely not work like that. Using a regular class would also work I guess, but I think it's better to define it as a STRUCTURE PCRYPT_CTX instead. That will not need initialization also.
Chris Pyrgas

XSharp Development Team test
chris(at)xsharp.eu
User avatar
baramuse
Posts: 87
Joined: Tue Nov 29, 2022 8:31 am
Location: France

A call to PInvoke function has unbalanced the stack

Post by baramuse »

Hi Chris,

actually I did init the class, just did not pasted it.
sorry I forget to paste the _DLL invoke code as :

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS BYTE[], count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

LOCAL ctx		AS PCRYPT_CTX
LOCAL bKey		AS BYTE[]
ctx := PCRYPT_CTX{}
bKey := BYTE[]{16}{235,89,56,22,81,127,158,80,216,87,210,149,134,206,99,129}
    
PCryptInit(ctx, bKey, bKey:Length)
I tried fist with the VOSTRUCT
Then I tried with a STRUCTURE but can't have fixed lenght arrays ?

Code: Select all

PUBLIC STRUCTURE PCRYPT_CTX
    PUBLIC DIM P[16+2]	AS DWORD
    PUBLIC DIM S[4*256]	AS DWORD
END STRUCTURE

==> Error	XS0573	'PCRYPT_CTX': cannot have instance property or field initializers in structs
User avatar
Chris
Posts: 4599
Joined: Thu Oct 08, 2015 7:48 am
Location: Greece

A call to PInvoke function has unbalanced the stack

Post by Chris »

Hi Basile,

Ah right, it's not easy to use fixed arrays within STRUCTUREs. It should be possible with the FIXED keyword, but this doesn't seem to work, will further look nto it.

But can you please post the original VO code that you were using and that was calling the original version of the function?
Chris Pyrgas

XSharp Development Team test
chris(at)xsharp.eu
User avatar
baramuse
Posts: 87
Joined: Tue Nov 29, 2022 8:31 am
Location: France

A call to PInvoke function has unbalanced the stack

Post by baramuse »

Sure :

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

STRUCTURE PCRYPT_CTX
	MEMBER DIM P[16+2]	AS DWORD
	MEMBER DIM S[4,256]	AS DWORD //That, I don't understand, looks like a 2-dimensional array instead of the 1-dim in the C struct ?

FUNCTION PCrypt(cStr AS STRING, cKey AS STRING, lEncrypt AS LOGIC) AS STRING PASCAL
LOCAL ctx		IS PCRYPT_CTX

PCryptInit(@ctx, String2Psz(cKey), INT(_CAST,SLen(cKey)))
...
User avatar
robert
Posts: 4290
Joined: Fri Aug 21, 2015 10:57 am
Location: Netherlands

A call to PInvoke function has unbalanced the stack

Post by robert »

Basile,
I think you are passing the structure on the stack, but the function expects the address of the structure on the stack.
The C code has this as first parameter:

Code: Select all

PCRYPT_CTX *ctx,
I think the best option is to use the VOSTRUCT.
You are not showing how you call it, but something like this should work.

Code: Select all

LOCAL ctx IS PCRYPT_CTX
PCryptInit(@ctx, String2Psz(cKey), (INT) SLen(cKey))
Alternatively you can declare the variable with 'AS' but then you would have to allocate memory and free that memory and pass the variable without the '@' operator.
Finally, make sure that if the C DLL is compiled for x86, that your program also runs in x86.
The only thing that I am not sure of is the size of BF_LONG. It is declared as "unsigned liong", but is that a 32 bit or 64 bit variable?
Robert
XSharp Development Team
The Netherlands
robert@xsharp.eu
User avatar
baramuse
Posts: 87
Joined: Tue Nov 29, 2022 8:31 am
Location: France

A call to PInvoke function has unbalanced the stack

Post by baramuse »

Robert,

that's excalty how it was called originally (from VOXPorter) :

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

VOSTRUCT PCRYPT_CTX
    MEMBER DIM P[16+2]	AS DWORD
    MEMBER DIM S[4*256]	AS DWORD

LOCAL ctx IS PCRYPT_CTX
PCryptInit(@ctx, String2Psz(cKey),  INT(_CAST,SLen(cKey)))
And was throwing exception.
Finally, make sure that if the C DLL is compiled for x86, that your program also runs in x86.
C DLL was compiled about 15 years ago, so pretty sure it was x86 :)
The XSharp program is targeting x86 as well
User avatar
Chris
Posts: 4599
Joined: Thu Oct 08, 2015 7:48 am
Location: Greece

A call to PInvoke function has unbalanced the stack

Post by Chris »

Hi Basile,

I suspect it has to do with the 2nd parameter, which expects a pointer to a buffer, but can you please send me (chris at xsharp eu) the dll to give it some tests here? Together with a small VO sample code of using it, the original one, not after VOXporter has maybe made any changes to it.
Chris Pyrgas

XSharp Development Team test
chris(at)xsharp.eu
User avatar
robert
Posts: 4290
Joined: Fri Aug 21, 2015 10:57 am
Location: Netherlands

A call to PInvoke function has unbalanced the stack

Post by robert »

Basile,

Did you try to declare the _DLL FUNCTION "AS VOID STRICT:PgsUti32.PCryptInit" ?

Robert
XSharp Development Team
The Netherlands
robert@xsharp.eu
User avatar
baramuse
Posts: 87
Joined: Tue Nov 29, 2022 8:31 am
Location: France

A call to PInvoke function has unbalanced the stack

Post by baramuse »

Hi Robert, I just tested it, reverted to the ported code only to change PASCAL to STRICT.. and it worked !!
Thanks A LOT !!
Post Reply