Page 1 of 2
A call to PInvoke function has unbalanced the stack
Posted: Tue Jun 13, 2023 7:21 am
by baramuse
Hi everyone,
in our VO app, we were using a C dll for encryption/decryption
The ported code seemto crash when calling the dll.
We do have the source code of the C dll so I tried to see what was bother the Xsharp version of the code :
Code: Select all
#define BF_LONG unsigned long
#define BF_ROUNDS 16
typedef struct bf_key_st
{
BF_LONG P[BF_ROUNDS+2];
BF_LONG S[4*256];
} BF_KEY;
typedef BF_KEY PCRYPT_CTX;
void PCryptInit(PCRYPT_CTX *ctx, unsigned char *key, int count)
{
BF_set_key(ctx, count, key);
return;
}
Now on the XSharp side here is the code that was generated by Xporter :
Code: Select all
_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit
VOSTRUCT PCRYPT_CTX
MEMBER DIM P[16+2] AS DWORD
MEMBER DIM S[4*256] AS DWORD
When using that I have a exception :
A call to PInvoke function has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature.
So I tried to match the signature "closer"
Code: Select all
_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS BYTE[], count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit
USING System.Runtime.InteropServices
[StructLayout(LayoutKind.Sequential)];
PUBLIC CLASS PCRYPT_CTX
PUBLIC DIM P[16+2] AS DWORD
PUBLIC DIM S[4*256] AS DWORD
END CLASS
LOCAL ctx AS PCRYPT_CTX
LOCAL bKey AS BYTE[]
bKey := BYTE[]{16}{235,89,56,22,81,127,158,80,216,87,210,149,134,206,99,129}
PCryptInit(ctx, bKey, bKey:Length)
Same exception, maybe I'm not looking in the right place ?
Any suggestion ?
A call to PInvoke function has unbalanced the stack
Posted: Tue Jun 13, 2023 7:46 am
by Chris
Hi Basile,
Can you please show also the exact code you were using to call the function with its original definition?
Also please note that in the modified version of the code, you are passing the first parameter "ctx" (which represents a regular type) uninitialized, so this will definitely not work like that. Using a regular class would also work I guess, but I think it's better to define it as a STRUCTURE PCRYPT_CTX instead. That will not need initialization also.
A call to PInvoke function has unbalanced the stack
Posted: Tue Jun 13, 2023 7:51 am
by baramuse
Hi Chris,
actually I did init the class, just did not pasted it.
sorry I forget to paste the _DLL invoke code as :
Code: Select all
_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS BYTE[], count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit
LOCAL ctx AS PCRYPT_CTX
LOCAL bKey AS BYTE[]
ctx := PCRYPT_CTX{}
bKey := BYTE[]{16}{235,89,56,22,81,127,158,80,216,87,210,149,134,206,99,129}
PCryptInit(ctx, bKey, bKey:Length)
I tried fist with the VOSTRUCT
Then I tried with a STRUCTURE but can't have fixed lenght arrays ?
Code: Select all
PUBLIC STRUCTURE PCRYPT_CTX
PUBLIC DIM P[16+2] AS DWORD
PUBLIC DIM S[4*256] AS DWORD
END STRUCTURE
==> Error XS0573 'PCRYPT_CTX': cannot have instance property or field initializers in structs
A call to PInvoke function has unbalanced the stack
Posted: Tue Jun 13, 2023 8:18 am
by Chris
Hi Basile,
Ah right, it's not easy to use fixed arrays within STRUCTUREs. It should be possible with the FIXED keyword, but this doesn't seem to work, will further look nto it.
But can you please post the original VO code that you were using and that was calling the original version of the function?
A call to PInvoke function has unbalanced the stack
Posted: Tue Jun 13, 2023 8:23 am
by baramuse
Sure :
Code: Select all
_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit
STRUCTURE PCRYPT_CTX
MEMBER DIM P[16+2] AS DWORD
MEMBER DIM S[4,256] AS DWORD //That, I don't understand, looks like a 2-dimensional array instead of the 1-dim in the C struct ?
FUNCTION PCrypt(cStr AS STRING, cKey AS STRING, lEncrypt AS LOGIC) AS STRING PASCAL
LOCAL ctx IS PCRYPT_CTX
PCryptInit(@ctx, String2Psz(cKey), INT(_CAST,SLen(cKey)))
...
A call to PInvoke function has unbalanced the stack
Posted: Tue Jun 13, 2023 8:46 am
by robert
Basile,
I think you are passing the structure on the stack, but the function expects the address of the structure on the stack.
The C code has this as first parameter:
I think the best option is to use the VOSTRUCT.
You are not showing how you call it, but something like this should work.
Code: Select all
LOCAL ctx IS PCRYPT_CTX
PCryptInit(@ctx, String2Psz(cKey), (INT) SLen(cKey))
Alternatively you can declare the variable with 'AS' but then you would have to allocate memory and free that memory and pass the variable without the '@' operator.
Finally, make sure that if the C DLL is compiled for x86, that your program also runs in x86.
The only thing that I am not sure of is the size of BF_LONG. It is declared as "unsigned liong", but is that a 32 bit or 64 bit variable?
Robert
A call to PInvoke function has unbalanced the stack
Posted: Tue Jun 13, 2023 8:54 am
by baramuse
Robert,
that's excalty how it was called originally (from VOXPorter) :
Code: Select all
_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit
VOSTRUCT PCRYPT_CTX
MEMBER DIM P[16+2] AS DWORD
MEMBER DIM S[4*256] AS DWORD
LOCAL ctx IS PCRYPT_CTX
PCryptInit(@ctx, String2Psz(cKey), INT(_CAST,SLen(cKey)))
And was throwing exception.
Finally, make sure that if the C DLL is compiled for x86, that your program also runs in x86.
C DLL was compiled about 15 years ago, so pretty sure it was x86
![Smile :)](https://dt9qzg9h465rt.cloudfront.net/phpBB3/images/smilies/icon_e_smile.gif)
The XSharp program is targeting x86 as well
A call to PInvoke function has unbalanced the stack
Posted: Wed Jun 14, 2023 6:34 am
by Chris
Hi Basile,
I suspect it has to do with the 2nd parameter, which expects a pointer to a buffer, but can you please send me (chris at xsharp eu) the dll to give it some tests here? Together with a small VO sample code of using it, the original one, not after VOXporter has maybe made any changes to it.
A call to PInvoke function has unbalanced the stack
Posted: Wed Jun 14, 2023 6:36 am
by robert
Basile,
Did you try to declare the _DLL FUNCTION "AS VOID STRICT:PgsUti32.PCryptInit" ?
Robert
A call to PInvoke function has unbalanced the stack
Posted: Wed Jun 14, 2023 7:08 am
by baramuse
Hi Robert, I just tested it, reverted to the ported code only to change PASCAL to STRICT.. and it worked !!
Thanks A LOT !!