www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Strange stack variable corruption error after calling extern(C)

reply cc <cc nevernet.com> writes:
Hello, I've been encountering a strange problem that seems to 
occur after calling some external C functions.  I've been working 
on a program that incorporates the FMOD C API for playing sound, 
with a simple D binding based off the C headers, and actually 
everything works more or less fine, I've had working sound in my 
program for a few months now.  However I recently started 
noticing some strange behavior, currently using DMD v2.070.2 
(haven't tried v2.071 yet, will soon).  I can't post the entire 
program but I'll include the relevant code, I might try to make a 
small working compilable sample if this isn't enough information.


// fmod/c/fmod.d:
// Adapted from C headers
extern(C):
struct FMOD_SYSTEM;
struct FMOD_SOUND;
enum FMOD_RESULT : int { ... }
enum FMOD_MODE : uint { ... }
FMOD_RESULT FMOD_System_CreateSound(FMOD_SYSTEM *system, const 
char *name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO 
*exinfo, FMOD_SOUND **sound);

// sound.d:
class Sound {
	static immutable string pathPrefix = "data/sound/";
	enum FMOD_MODE mode = FMOD_MODE.FMOD_LOOP_OFF | 
FMOD_MODE.FMOD_3D | FMOD_MODE.FMOD_3D_HEADRELATIVE | 
FMOD_MODE.FMOD_3D_LINEARSQUAREROLLOFF;

	FMOD_SOUND *snd;
	FMOD_CHANNEL *channel;
	string name;

	void load() {
		string fullpath = pathPrefix ~ name ~ ".wav";

		writefln("loading sound before: <%s> <%s> [%08x] <%s>", 
pathPrefix, name, fullpath.ptr, fullpath);
		FMOD_System_CreateSound(system, fullpath.toStringz, mode, null, 
&snd);
		writefln("A loading sound after: <%s> <%s> [%08x] <%s>", 
pathPrefix, name, fullpath.ptr, fullpath);
		writefln("B loading sound after: <%s> <%s> [%08x] <%s>", 
pathPrefix, name, fullpath.ptr, fullpath);
		if (1) {
			writefln("C loading sound after: <%s> <%s> [%08x] <%s>", 
pathPrefix, name, fullpath.ptr, fullpath);
		}
	}
}

// Output:
loading sound before: <data/sound/> <slash0> [04368a40] 
<data/sound/slash0.wav>
A loading sound after: <data/sound/> <slash0> [04368a40] 
<data/sound/slash0.wav>
B loading sound after: <data/sound/> <slash0> [04368a40] 
<data/sound/slash0.wav>
C loading sound after: <data/sound/> <slash0> [005df255] <C 
loading sound after: <%s> <%s> [%08x] <%s>>


As you can see I concatenate the path, filename, and extension 
strings, simple enough, then pass the result of .toStringz to the 
function.  What happens next is the bizarre behavior to the 
fullstring variable: Everything is fine until the program enters 
the if check, then for some reason the value of fullpath changes 
to be that of the string literal being passed to writefln().  If 
I try to do something else such as save the value of fullpath as 
a class member variable first, I get some other problem like 
object.Error (0): Access Violation when it comes time to print 
the string.  The sound itself still plays fine, but something is 
getting corrupted by something, somewhere, and I'm completely at 
a loss as to understand what's going on.  Does anyone have any 
advice on what's happening here?
May 03 2016
parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
On Tuesday, 3 May 2016 at 11:32:31 UTC, cc wrote:
 Hello, I've been encountering a strange problem that seems to 
 occur after calling some external C functions.  I've been 
 working on a program that incorporates the FMOD C API for 
 playing sound, with a simple D binding based off the C headers, 
 and actually everything works more or less fine, I've had 
 working sound in my program for a few months now.  However I 
 recently started noticing some strange behavior, currently 
 using DMD v2.070.2 (haven't tried v2.071 yet, will soon).  I 
 can't post the entire program but I'll include the relevant 
 code, I might try to make a small working compilable sample if 
 this isn't enough information.

 [...]
It seems that one of the fmod functions you declared is not correct. Either the fmod api is not using the c calling convention or you made a mistake when declaring the paramters of the fmod functions. You should double check that the functions match the fmod headers. Kind Regards Benjamin Thaut
May 03 2016
parent reply cc <cc nevernet.com> writes:
On Tuesday, 3 May 2016 at 12:48:37 UTC, Benjamin Thaut wrote:
 It seems that one of the fmod functions you declared is not 
 correct. Either the fmod api is not using the c calling 
 convention or you made a mistake when declaring the paramters 
 of the fmod functions. You should double check that the 
 functions match the fmod headers.
I see, thanks. Double checking the original headers, I see the function is defined as: FMOD_RESULT F_API FMOD_System_CreateSound (FMOD_SYSTEM *system, const char *name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO *exinfo, FMOD_SOUND **sound); F_API seems to be defined as follows: ... #elif defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(_WIN64) #define F_STDCALL _stdcall #define F_DECLSPEC __declspec #define F_DLLEXPORT ( dllexport ) ... #ifdef DLL_EXPORTS #define F_API F_DECLSPEC F_DLLEXPORT F_STDCALL #else #define F_API F_STDCALL #endif I tried going by the information suggested on this page: http://wiki.dlang.org/D_binding_for_C#cdecl.2C_pascal.2C_stdcall but if I declare the function, in D, as: extern(Windows) FMOD_RESULT FMOD_System_CreateSound (FMOD_SYSTEM *system, const char *name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO *exinfo, FMOD_SOUND **sound); it fails to link with "Error 42: Symbol Undefined _FMOD_System_CreateSound 20". With extern(C) it compiles and runs but the problem from above persists.
May 03 2016
next sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
On Tuesday, 3 May 2016 at 19:06:30 UTC, cc wrote:
 it fails to link with "Error 42: Symbol Undefined 
 _FMOD_System_CreateSound 20".  With extern(C) it compiles and 
 runs but the problem from above persists.
Is this on Windows x64? Try replacing FMOD_RESULT by int. When declaring the fmod create sound function and see if that helps.
May 04 2016
parent reply cc <cc nevernet.com> writes:
On Wednesday, 4 May 2016 at 09:40:55 UTC, Benjamin Thaut wrote:
 On Tuesday, 3 May 2016 at 19:06:30 UTC, cc wrote:
 it fails to link with "Error 42: Symbol Undefined 
 _FMOD_System_CreateSound 20".  With extern(C) it compiles and 
 runs but the problem from above persists.
Is this on Windows x64? Try replacing FMOD_RESULT by int. When declaring the fmod create sound function and see if that helps.
The OS is Win64 though the program is being compiled as 32-bit and I'm using the 32-bit distributed DLL. fmod.dll: PE32 executable (DLL) (GUI) Intel 80386, for MS Windows Tried int and long as the return type, same issue both ways. Tried void too just in case, same thing though.
May 04 2016
parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
On Wednesday, 4 May 2016 at 17:53:32 UTC, cc wrote:
 The OS is Win64 though the program is being compiled as 32-bit 
 and I'm using the 32-bit distributed DLL.
 fmod.dll: PE32 executable (DLL) (GUI) Intel 80386, for MS 
 Windows

 Tried int and long as the return type, same issue both ways.  
 Tried void too just in case, same thing though.
Could you please post the definition of FMOD_RESULT. Its possible that the create sound function returns it on stack and not inside a register. This is usually the case if FMOD_RESULT is defined as a struct in C/C++. But its hard to say. In your case I would actually look at the disassembly and step through it to see where its going wrong and messing up the stack.
May 05 2016
parent cc <cc nevernet.com> writes:
On Thursday, 5 May 2016 at 09:42:00 UTC, Benjamin Thaut wrote:
 On Wednesday, 4 May 2016 at 17:53:32 UTC, cc wrote:
 The OS is Win64 though the program is being compiled as 32-bit 
 and I'm using the 32-bit distributed DLL.
 fmod.dll: PE32 executable (DLL) (GUI) Intel 80386, for MS 
 Windows

 Tried int and long as the return type, same issue both ways.  
 Tried void too just in case, same thing though.
Could you please post the definition of FMOD_RESULT. Its possible that the create sound function returns it on stack and not inside a register. This is usually the case if FMOD_RESULT is defined as a struct in C/C++. But its hard to say. In your case I would actually look at the disassembly and step through it to see where its going wrong and messing up the stack.
In fmod_common.h: typedef enum { FMOD_OK, /* No errors. */ FMOD_ERR_ALREADYLOCKED, /* Tried to call lock a second time before unlock was called. */ ... FMOD_RESULT_FORCEINT = 65536 /* Makes sure this enum is signed 32bit. */ } FMOD_RESULT; In D I defined it as enum FMOD_RESULT : int { ... }
May 05 2016
prev sibling parent reply Lewis <musicaljelly gmail.com> writes:
On Tuesday, 3 May 2016 at 19:06:30 UTC, cc wrote:
 I see, thanks.  Double checking the original headers, I see the 
 function is defined as:

 FMOD_RESULT F_API FMOD_System_CreateSound               
 (FMOD_SYSTEM *system, const char *name_or_data, FMOD_MODE mode, 
 FMOD_CREATESOUNDEXINFO *exinfo, FMOD_SOUND **sound);


 F_API seems to be defined as follows:

 ...
 #elif defined(WIN32) || defined(_WIN32) || defined(__WIN32__) 
 || defined(_WIN64)
     #define F_STDCALL _stdcall
     #define F_DECLSPEC __declspec
     #define F_DLLEXPORT ( dllexport )
 ...
 #ifdef DLL_EXPORTS
     #define F_API F_DECLSPEC F_DLLEXPORT F_STDCALL
 #else
     #define F_API F_STDCALL
 #endif


 I tried going by the information suggested on this page: 
 http://wiki.dlang.org/D_binding_for_C#cdecl.2C_pascal.2C_stdcall
 but if I declare the function, in D, as:
 extern(Windows) FMOD_RESULT FMOD_System_CreateSound
   (FMOD_SYSTEM *system, const char *name_or_data, FMOD_MODE 
 mode, FMOD_CREATESOUNDEXINFO *exinfo, FMOD_SOUND **sound);

 it fails to link with "Error 42: Symbol Undefined 
 _FMOD_System_CreateSound 20".  With extern(C) it compiles and 
 runs but the problem from above persists.
Holy crap, thank you. I know this is late, but I was playing around with derelictFMOD, and ran into a strange crash like yours on shutdown. Looking at the disassembly revealed that FMOD_System_Close() was popping more off the stack as it finished than I would have expected it to. I tried changing the calling convention in derelictFMOD to extern(Windows) as mentioned here, and that fixed the problem. I'll submit a pull request to that project to get that fixed. But thank you both for having this forum discussion 11 months prior that would eventually tip me in the direction of the solution :)
Apr 14
parent reply cc <cc nevernet.com> writes:
On Saturday, 15 April 2017 at 00:23:42 UTC, Lewis wrote:
 Holy crap, thank you. I know this is late, but I was playing 
 around with derelictFMOD, and ran into a strange crash like 
 yours on shutdown. Looking at the disassembly revealed that 
 FMOD_System_Close() was popping more off the stack as it 
 finished than I would have expected it to. I tried changing the 
 calling convention in derelictFMOD to extern(Windows) as 
 mentioned here, and that fixed the problem.

 I'll submit a pull request to that project to get that fixed. 
 But thank you both for having this forum discussion 11 months 
 prior that would eventually tip me in the direction of the 
 solution  :)
How did you get around the linker error? I was never able to get this working, I tried pragma(mangle, "FMOD_System_CreateSound") extern(Windows) FMOD_RESULT FMOD_System_CreateSound (FMOD_SYSTEM *system, const char *name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO *exinfo, FMOD_SOUND **sound); but it still prepends the underbar and fails to link with undefined symbol.
Apr 15
parent reply cc <cc nevernet.com> writes:
Ok, I took another stab at this since I've had the problem 
sitting for however many months and I think I finally got it 
figured out.  I needed to reimport the import library from the 
DLL (via implib.exe) WITHOUT the /system switch, then, on 
inspecting it, it appears the correct function names to use are 
FMOD5_* rather than FMOD_*.

So,

FMOD_RESULT FMOD_System_CreateSound(FMOD_SYSTEM *system, const 
char *name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO 
*exinfo, FMOD_SOUND **sound);

becomes

pragma(mangle, "FMOD5_System_CreateSound")
FMOD_RESULT FMOD_System_CreateSound(FMOD_SYSTEM *system, const 
char *name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO 
*exinfo, FMOD_SOUND **sound);

or this would also work of course:

FMOD_RESULT FMOD5_System_CreateSound(FMOD_SYSTEM *system, const 
char *name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO 
*exinfo, FMOD_SOUND **sound);
alias FMOD5_System_CreateSound FMOD_System_CreateSound;

Now it finally links and the stack is no longer being corrupted!
Apr 16
parent reply cc <cc nevernet.com> writes:
All this with extern(Windows) rather than extern(C) by the way.
Apr 16
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 16 April 2017 at 08:34:12 UTC, cc wrote:
 All this with extern(Windows) rather than extern(C) by the way.
Why not use loadLibraryA ? then all the problems go away :) this is how derelict does it as well.
Apr 16