SRL Archive - Lape

Weird bug with the plugin


CynicRus

2014-09-26 05:21:23

Hey guys. I think, I found the Bug in Simba. When I trying return the value of the function of the delphi plugin, I always get Exception in Script: Runtime error: "Access violation" at line 11, column 10.
When I try to debug Simba with this plugin, I get External:Sigsegv in ffi_call(FCif,
Func, Res, Args); of the lpffi unit, line 418. Args is empty.

I assume, this is mostly the lape bug, so I am writing this message here.

I use Delphi XE3, plugin code is:



uses
System.SysUtils,
System.Classes,Windows,VCL.Dialogs,
{ uMSScript in 'uMSScript.pas', }
math;

var
memisset: Boolean = False;
OldMemoryManager: TMemoryManager;

function GetPluginABIVersion: Integer; Cdecl;export;
begin
Result := 2;
end;

procedure SetPluginMemManager(MemMgr : TMemoryManager); Cdecl;export;
begin
if memisset then
exit;
GetMemoryManager(OldMemoryManager);
SetMemoryManager(MemMgr);
memisset := true;
end;

procedure OnDetach; Cdecl; export;
begin
SetMemoryManager(OldMemoryManager);
end;

function Hello(S: AnsiString):AnsiString;cdecl;
begin
result:=S;
end;

function HelloAgain():AnsiString;cdecl;
begin
result:='HelloAgain';
end;

function GetFunctionCount(): Integer; Cdecl; export;
begin
Result := 2;
end;

function GetFunctionInfo(x: Integer; var ProcAddr: Pointer; var ProcDef: PAnsiChar): Integer;Cdecl; export;
begin
case x of
0:
begin
ProcAddr := [MENTION=7782]Hello[/MENTION];
StrPCopy(ProcDef, 'function Hello(S: String):String;');
end;
1:
begin
ProcAddr := [MENTION=7782]Hello[/MENTION]Again;
StrPCopy(ProcDef, 'function HelloAgain():String;');
end;
else
x := -1;
end;

Result := x;
end;

exports GetPluginABIVersion;
exports SetPluginMemManager;
{exports GetTypeCount;
exports GetTypeInfo;}
exports GetFunctionCount;
exports GetFunctionInfo;
exports OnDetach;

begin
end.


Test script:

program new;
{$loadlib mylib.dll}

procedure TestPlug;
begin

WriteLn(Hello('simba -=> delphi'));
WriteLn(HelloAgain);

end;
begin
TestPlug;
end.

Output:
Compiled successfully in 343 ms.
simba -=> delphi
Exception in Script: Runtime error: "Access violation" at line 8, column 10
Simba version is 1.0.4, heh.

Cheers,Cynic.

CynicRus

2014-09-26 06:38:39

function HelloAgain():PAnsiChar;cdecl;
begin
result:='HelloAgain';
end;

- same results.

Brandon

2014-09-26 06:39:51

ProcAddr := @Hello Again;


says that you have a space between 'hello' and 'again' or some weird character there.. Otherwise the mention tags would not separate them if it was one word. It'd show up as mention: HelloAgain instead of mention: Hello Again.

Try naming the function something different and also try doing:

writeln(HelloAgain()) instead of writeln(HelloAgain) and see what happens? Does it work if you export both Hello and HelloAgain? Does it work in PS? It should I guess.. I can't see anything wrong with your code so I guess it is a bug.

CynicRus

2014-09-26 07:58:59

It's isn't work, if I change any value and return it from the function-(

slacky

2014-09-26 14:08:12

if you have the latest Simba, aka any 1.1RC then you can try this:

library Test;

uses
System.SysUtils,
System.Classes,
Windows,
math;


var
memisset: Boolean = False;
OldMemoryManager: TMemoryManager;


type
PParamArray = ^TParamArray;
TParamArray = array[Word] of Pointer;


procedure Hello(const Args:PParamArray; const Result:Pointer); cdecl; export;
begin
PAnsiString(Result)^ := PAnsiString(Args^[0])^;
end;

procedure DoSomething(const Args:PParamArray; const Result:Pointer); cdecl; export;
begin
PInteger(Result)^ := 10*5+1;
end;

procedure HelloWorld(const Args:PParamArray; const Result:Pointer); cdecl; export;
begin
PAnsiString(Result)^ := 'hello world';
end;

//------------------------------------------------------------
function GetPluginABIVersion: Integer; cdecl; export;
begin
Result := 2;
end;

procedure SetPluginMemManager(MemMgr : TMemoryManager); cdecl; export;
begin
if memisset then
exit;
GetMemoryManager(OldMemoryManager);
SetMemoryManager(MemMgr);
memisset := true;
end;

procedure OnDetach; Cdecl; export;
begin
SetMemoryManager(OldMemoryManager);
end;

function GetFunctionCount(): Integer; cdecl; export;
begin
Result := 3;
end;

function GetFunctionInfo(x: Integer; var ProcAddr: Pointer; var ProcDef: PAnsiChar): Integer; cdecl; export;
begin
case x of
0:
begin
ProcAddr := [MENTION=7782]Hello[/MENTION];
StrPCopy(ProcDef, 'function Hello(S: String):String; native;');
end;
1:
begin
ProcAddr := [MENTION=7782]DoSomething[/MENTION];
StrPCopy(ProcDef, 'function DoSomething(): Integer; native;');
end;
2:
begin
ProcAddr := [MENTION=1582]helloworld[/MENTION];
StrPCopy(ProcDef, 'function HelloWorld(): String; native;');
end;
else
x := -1;
end;

Result := x;
end;

exports GetPluginABIVersion;
exports SetPluginMemManager;
exports GetFunctionCount;
exports GetFunctionInfo;
exports OnDetach;


end.
If this fails then the issue probably not in the FFI, as none of the exported functions will be handled by lapes FFI in this case.

Tho I am really not sure if it will help at all… I have not looked in to the issue you have what so ever :P

CynicRus

2014-09-26 15:18:15

Same result, DoSomething return 0. This is weird.

Brandon

2014-09-26 15:44:00

From: CynicRus
Same result, DoSomething return 0. This is weird.



Run this:


{$loadlib Cynic}

begin
writeln(@Hello);
writeln(@HelloAgain);
end.

It should print something like:


function(AnsiString):AnsiString ("Hello"::0x32B0008)
function():AnsiString ("HelloAgain"::0x32B0050)

Does it print NIL for "HelloAgain"? Does it print anything?


I compiled your code http://paste.villavu.com/raw/hj8RLPXnghv3pNzxpHWD/.
I used Lazarus-1.2.2 & FPC-2.6.4-rev44748.
I was unable to reproduce the problem you are having.
I would probably have to install Delphi XE3 to reproduce it.. It seems like a compiler problem..?


CynicRus

2014-09-26 16:19:30

From: Brandon
Run this:


{$loadlib Cynic}

begin
writeln(@Hello);
writeln(@HelloAgain);
end.

It should print something like:


function(AnsiString):AnsiString ("Hello"::0x32B0008)
function():AnsiString ("HelloAgain"::0x32B0050)

Does it print NIL for "HelloAgain"? Does it print anything?


I compiled your code http://paste.villavu.com/raw/hj8RLPXnghv3pNzxpHWD/.
I used Lazarus-1.2.2 & FPC-2.6.4-rev44748.
I was unable to reproduce the problem you are having.
I would probably have to install Delphi XE3 to reproduce it.. It seems like a compiler problem..?


Well, I run that as you say:

Compiled successfully in 328 ms.
function(AnsiString):AnsiString ("Hello"::0x780D610)
function():Int32 ("DoSomething"::0x780D624)
function():AnsiString ("HelloWorld"::0x780D634)
Successfully executed.
Of course In Lazarus I do not have any problem, but as I said - I try to use unicode version of Delphi. It is not clear why it is possible to write a plugin in C/C++, but cannot in Delphi. I think, code in Delphi 7 also will work without any problem.

UPD: Same results in Delphi2010.

Upd1: In Delphi 2006 working only Hello and DoSomething procedure:)

Delphi2006 results:

Compiled successfully in 359 ms.
Hi!
51
Error: Access violation at line 11
Execution failed.

Compiled version of dll in the attachment.

Brandon

2014-09-26 17:36:04

From: CynicRus
Well, I run that as you say:

Compiled successfully in 328 ms.
function(AnsiString):AnsiString ("Hello"::0x780D610)
function():Int32 ("DoSomething"::0x780D624)
function():AnsiString ("HelloWorld"::0x780D634)
Successfully executed.
Of course In Lazarus I do not have any problem, but as I said - I try to use unicode version of Delphi. It is not clear why it is possible to write a plugin in C/C++, but cannot in Delphi. I think, code in Delphi 7 also will work without any problem.

UPD: Same results in Delphi2010.

Upd1: In Delphi 2006 working only Hello and DoSomething procedure:)

Compiled version of dll in the attachment.



There is something wrong with your compiler or plugins.. NOTHING works in C or C++.. Don't have to print anything. Just calling the functions alone crashes.


Test code:

#include <windows.h>
#include <iostream>

std::string ErrorMessage(std::uint32_t Error)
{
LPTSTR lpMsgBuf = nullptr;
FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, nullptr, Error, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), reinterpret_cast<LPTSTR>(&lpMsgBuf), 0, nullptr);
std::string res(lpMsgBuf);
LocalFree(lpMsgBuf);
return res;
}

typedef struct
{
int32_t ref;
int32_t length;
char data[1];
} delphi_string;

int main()
{
HMODULE mod = LoadLibrary("test.dll");
if (mod)
{
int (__cdecl *GetFunctionCount)() = (int (__cdecl *)())GetProcAddress(mod, "GetFunctionCount");
int (__cdecl *GetFunctionInfo)(int, void*&, char*&) = (int (__cdecl *)(int, void*&, char*&))GetProcAddress(mod, "GetFunctionInfo");

std::cout<<"Function count: "<<GetFunctionCount()<<"\n";

const char* (__cdecl *hello)(const char*) = NULL;
int (__cdecl *DoSomething)() = NULL;
char* (__cdecl *HelloWorld)() = NULL;

char* dummy = new char[256];
GetFunctionInfo(0, (void*&)hello, dummy);
std::cout<<dummy<<"\n";
GetFunctionInfo(1, (void*&)DoSomething, dummy);
std::cout<<dummy<<"\n";
GetFunctionInfo(2, (void*&)HelloWorld, dummy);
std::cout<<dummy<<"\n";
delete[] dummy;


DoSomething(); //ERROR.. Crashes with a segfault. Hello crashes. HelloWorld crashes too.



/*char* foo = new char[256];
delphi_string* str = (delphi_string*)&foo[0];

strcpy(str->data, "hello");
str->ref = 0xFF;
str->length = strlen(str->data);
str->data[str->length + 1] = 0;

hello(foo);

delete[] foo;*/



FreeLibrary(mod);
}
else
{
std::cout<<ErrorMessage(GetLastError());
std::cout<<"Failed to load module.";
}

return 0;
}


Results (before it crashes):

Function count: 3
function Hello(S: String):String; native;
function DoSomething(): Integer; native;
function HelloWorld(): String; native;

Process returned -1073741819 (0xC0000005)

0xC0000005 = Access Violation.


You might have to export the above functions. I tried it in Simba and I tried it in C & C++. The first function works in Simba but after that, you're out of luck.



I checked your functions' assembly.. Everything is fine.. Though, it's not in Unicode as you said.. It's exactly how you coded it (using AnsiString). The assembly is perfectly fine so there's something else wrong. Perhaps it HAS to be in the export table which is the only logical conclusion I can come up with.


The ASM is below:


;This is your HELLO Function.
;------------------------------------------------------------------------------
L004CD610:
push ebp
mov ebp,esp
mov eax,[ebp+0Ch]
mov edx,[ebp+08h]
mov edx,[edx]
mov edx,[edx]
call SUB_L00408F04
pop ebp
retn




;This is your DO_SOMETHING Function.
;------------------------------------------------------------------------------
L004CD624:
push ebp
mov ebp,esp
mov eax,[ebp+08h]
mov dword ptr [eax],00000033h ;10*5+1 = 51-dec => 0x33h
pop ebp
retn


;This is your HELLO_WORLD Function.
;------------------------------------------------------------------------------
Align 4
L004CD634:
push ebp
mov ebp,esp
mov eax,[ebp+08h]
mov edx,SLP004CD654_hello_world
call SUB_L00408F04
pop ebp
retn


;The .data/.text section that contains global variables..
;It shows that it contains your 'hello world' string as an array of bytes..
;NOT wide-chars (aka Unicode-UTF16).
;It is encoded in ANSI string format (ASCII aka Windows-1252 on Windows.. UTF-8 on Linux).
;------------------------------------------------------------------------------
db 00h;
db 00h;
db E3h; '£'
db 04h;
db 01h;
db 00h;
dd FFFFFFFFh
dd 0000000Bh
SLP004CD654_hello_world:
db 'hello world'
Align 4


;------------------------------------------------------------------------------
GetPluginABIVersion:
mov eax,00000002h ;Return CDECL abi($2)
retn


;------------------------------------------------------------------------------
Align 4
SetPluginMemManager:
push ebp
mov ebp,esp
cmp byte ptr [L004D56FC],00h
jnz L004CD68D
mov eax,L004DB008
call SUB_L00406428
lea eax,[ebp+08h]
call SUB_L00406444
mov byte ptr [L004D56FC],01h
L004CD68D:
pop ebp
retn


;------------------------------------------------------------------------------
Align 4
OnDetach:
mov eax,L004DB008
call SUB_L00406444
retn


;------------------------------------------------------------------------------
Align 4
GetFunctionCount:
mov eax,00000003h
retn


;In the below:
;call SUB_L0041E50C ;is a call to StrPCopy.
;jmp L004CD70A ;is to "return". Aka "result := x".
;------------------------------------------------------------------------------
Align 4
GetFunctionInfo:
push ebp
mov ebp,esp
push ebx
mov ebx,[ebp+08h]
mov eax,ebx
sub eax,00000001h
jc L004CD6B9
jz L004CD6D3
dec eax
jz L004CD6ED
jmp L004CD707
L004CD6B9:
mov eax,[ebp+0Ch]
mov dword ptr [eax],L004CD610
mov eax,[ebp+10h]
mov eax,[eax]
mov edx,SLP004CD71C_function_Hello_S__String__String
call SUB_L0041E50C
jmp L004CD70A
L004CD6D3:
mov eax,[ebp+0Ch]
mov dword ptr [eax],L004CD624
mov eax,[ebp+10h]
mov eax,[eax]
mov edx,SLP004CD754_function_DoSomething____Integer_
call SUB_L0041E50C
jmp L004CD70A
L004CD6ED:
mov eax,[ebp+0Ch]
mov dword ptr [eax],L004CD634
mov eax,[ebp+10h]
mov eax,[eax]
mov edx,SLP004CD78C_function_HelloWorld____String__n
call SUB_L0041E50C
jmp L004CD70A
L004CD707:
or ebx,FFFFFFFFh

L004CD70A:
mov eax,ebx
pop ebx
pop ebp
retn


;The .data/.text section containing your strings that are used within the GetFunctionInfo function.
;Nothing to see here. It's plain text. No Unicode (UTF-16) here.
;------------------------------------------------------------------------------
db 00h;
db E3h; '£'
db 04h;
db 01h;
db 00h;
dd FFFFFFFFh
dd 00000029h
SLP004CD71C_function_Hello_S__String__String:
db 'function Hello(S: String):String; native;'
db 00h;
db 00h;
db 00h;
db E3h; '£'
db 04h;
db 01h;
db 00h;
dd FFFFFFFFh
dd 00000028h
SLP004CD754_function_DoSomething____Integer_:
db 'function DoSomething(): Integer; native;'
db 00h;
db 00h;
db 00h;
db 00h;
db E3h; '£'
db 04h;
db 01h;
db 00h;
dd FFFFFFFFh
dd 00000026h
SLP004CD78C_function_HelloWorld____String__n:
db 'function HelloWorld(): String; native;'
Align 4
unit00000028.Finalization:
retn


If it cannot load in C, I highly doubt it's going to load anywhere else. Once you do that and compile it, I'd look it over and see how things go if you want.

CynicRus

2014-09-26 17:43:12

I mean Delphi XE is unicode compiler, I used AnsiString only for Compability with LAzarus String. Ty @Brandon, I will try it!

Update: tried. Fail. In Delphi2006. Thats very weird. I've added functions to the export section, also I include the compiled dll to the topic.

@Brandon; in test1.zip - classic version of the plugin. This working as DLL perfectly. As Plugin - fail:)

Brandon

2014-09-26 21:04:02

From: CynicRus
..


Sorry for the very long explanation in advance.. There is a TLDR at the bottom if you don't care much about the details..


Neither of those plugins work. However, I solved the problem.. There's more than one actually but the memory manager causes the MOST problems of them all..


There's four things to note.

1. Delphi strings are all UTF-16 encoded on Windows unless explicitly specified otherwise (YOU MUST use AnsiString which you already figured out). Using wide-strings (delphi's native string on windows is UTF-16) in Simba is a NO-NO! You'll be as annoyed as I am.
2. Delphi strings have a weird import dependency on: BORLNDMM.DLL for EXPORTED functions!
3. You may use ShortStrings instead but there's a limitation of a maximum of 0xFF (255) characters.
4. I could not find a way to statically link to the above. It depends on the ShareMem header. It says that in the IDE itself but I could not find a static lib. (see below.. The code has a huge comment on this)



This is what you get when you try to load your modules in C without shipping the above module and ALL of its dependencies as well:



Now how do we get around it? We can ship your plugin with that module.. But that's annoying as hell.. We can avoid delphi strings and export using pchar..

OR we can get rid of your memory manager. Now I'm not an expert on Simba's memory manager and I have no clue why you use it, as most plugins I write is in C, C++ or Assembly. I did write the Direct-X plugin in Pascal without a memory manager and it seems to work perfectly fine.

IF you can get rid of it (the memory manager), this plugin will work.. I know because I tried it:



library Project1;

{ Important note about DLL memory management: ShareMem must be the
first unit in your library's USES clause AND your project's (select
Project-View Source) USES clause if your DLL exports any procedures or
functions that pass strings as parameters or function results. This
applies to all strings passed to and from your DLL--even those that
are nested in records and classes. ShareMem is the interface unit to
the BORLNDMM.DLL shared memory manager, which must be deployed along
with your DLL. To avoid using BORLNDMM.DLL, pass string information
using PChar or ShortString parameters. }

uses
//ShareMem,
System.SysUtils,
System.Classes,
Windows,
VCL.Dialogs,
math;

{$R *.res}

function GetPluginABIVersion: Integer; Cdecl;export;
begin
Result := 2;
end;

function Hello(S: AnsiString): AnsiString; cdecl;
begin
result := S;
end;

function HelloAgain(): AnsiString; cdecl;
begin
result := 'hello again';
end;

function GetFunctionCount(): Integer; Cdecl; export;
begin
Result := 2;
end;

function GetFunctionInfo(x: Integer; var ProcAddr: Pointer; var ProcDef: PAnsiChar): Integer;Cdecl; export;
begin
case x of
0:
begin
ProcAddr := [MENTION=7782]Hello[/MENTION];
StrPCopy(ProcDef, 'function Hello(S: string): string;');
end;
1:
begin
ProcAddr := [MENTION=7782]Hello[/MENTION]Again;
StrPCopy(ProcDef, 'function HelloAgain(): string;');
end;
else
x := -1;
end;

Result := x;
end;

exports GetPluginABIVersion;
exports GetFunctionCount;
exports GetFunctionInfo;

begin
end.


If you want to use your own memory manager then you need to allocate the strings yourself! AND free them yourself.. Don't ask me why. I have absolutely no idea what the memory manager does but it works when I allocate it myself. It's best to honestly just get rid of it and NOT handle the memory yourself.


IF you decide to use a memory manager (which I would avoid) then you can do it as follows:


type
PUInt32 = ^UInt32;

Function LenPtr(P: Pointer): PUInt32;
Begin
Result := PUInt32(PAnsiChar(P) - sizeof(Integer)); //pointer to the string's length.
End;

Function RefPtr(P: Pointer): PUInt32;
Begin
Result := PUInt32(PAnsiChar(P) - (sizeof(Integer) * 2)); //pointer to the string's reference count.
End;

function make_ps_string(S: PAnsiChar): Pointer;
var
Arr: ^AnsiChar;
I: Integer;
begin
Arr := AllocMem(Length(S) + 1); //Change the length to Length of S + 1 for double null termination (as required by PS).
RefPtr(@Arr[0])^ := 2; //Change the string's reference count to 2.

For I := 0 To Length(S) - 1 Do
Begin
Arr[I] := S^;
S := S + 1;
End;

Result := @Arr[0];
end;



function Hello(S: PAnsiChar): PAnsiChar;cdecl;
begin
result := S;
end;

function HelloAgain(): AnsiString; cdecl;
begin
result := make_ps_string('hello'); //translate a delphi string to a pascal one.
end;

This way, pascal has a reference to a string in raw memory. You will need to call FreeMem on the resulting string!



TLDR: Ditch the memory manager or find a fix/replacement for it. If I knew what the memory manager was for, I could perhaps help you with that but my knowledge there is limited.

Again, get rid of the memory manager code and your plugin will work fine (You don't have to export the functions either). This will save you the trouble for now.

I tested all of the above for hours in Delphi XE7. I'm fairly certain there are no other solutions.

CynicRus

2014-09-27 02:37:35

@Brandon; thank you for the explanation! Now problem is clear.

Wizzup?

2014-09-27 06:28:12

From: Brandon

TLDR: Ditch the memory manager or find a fix/replacement for it. If I knew what the memory manager was for, I could perhaps help you with that but my knowledge there is limited.

Again, get rid of the memory manager code and your plugin will work fine (You don't have to export the functions either). This will save you the trouble for now.

I tested all of the above for hours in Delphi XE7. I'm fairly certain there are no other solutions.


The functions for setting a memory manager are completely Free Pascal specific.
They are meant to make it possible to pass around arrays and other types
natively, without too much of a hassle. Even having much different compiler
versions from the one Simba was compiled with may already pose a problem.

So don't try to use it with Delphi, or other non-FPC languages. With those,
you're still stuck with the usual managed memory problems inherent (as a
feature) in languages like FPC.