www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - D-DLLs & Python

reply "Chris" <wendlec tcd.ie> writes:
I have written a DLL that I load into a Python program. 
Everything works fine (DLL is loaded via ctypes, functions can be 
called and are executed). Only the string handling is giving me a 
bit of a headache. The string in D is always complete garbage. I 
have written Python modules in D (with C wrappers) before and got 
them to work. But the DLL in D seems to be a completely different 
beast. Does anyone have experience with it?

Python uses ctypes, e.g.

myDLL = CDLL("myDLL") / WinDLL("myDLL")
myDLL.printThis(c_char_p("Hello world")) / 
myDLL.printThis(create_string_buffer("Hello world"))

The D side looks like like this:

export void printThis(ref char[] str) {
    printf("%s\n", str); // prints "Hello World"
    writeln(str); // prints garbage
}

OR

export void printThis(char* str) {
     printf("%s\n", str); // prints garbage
     writeln(str); // prints garbage
}

What am I doing wrong / missing here? I guess it has something to 
do with the pointers.

Thanks!

prin
Feb 19 2013
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 19 February 2013 at 16:23:45 UTC, Chris wrote:
 I have written a DLL that I load into a Python program. 
 Everything works fine (DLL is loaded via ctypes, functions can 
 be called and are executed). Only the string handling is giving 
 me a bit of a headache. The string in D is always complete 
 garbage. I have written Python modules in D (with C wrappers) 
 before and got them to work. But the DLL in D seems to be a 
 completely different beast. Does anyone have experience with it?

 Python uses ctypes, e.g.

 myDLL = CDLL("myDLL") / WinDLL("myDLL")
 myDLL.printThis(c_char_p("Hello world")) / 
 myDLL.printThis(create_string_buffer("Hello world"))

 The D side looks like like this:

 export void printThis(ref char[] str) {
    printf("%s\n", str); // prints "Hello World"
    writeln(str); // prints garbage
 }

 OR

 export void printThis(char* str) {
     printf("%s\n", str); // prints garbage
     writeln(str); // prints garbage
 }

 What am I doing wrong / missing here? I guess it has something 
 to do with the pointers.

 Thanks!

 prin
D doesn't use null termination for it's strings, strings are immutable(char)[]. You can form a D slice from a pointer by going slice = ptr[0..length] where length is the length of the array the pointer represents. You can't just take a c style string and expect writeln to work with it. Also, I think you should have extern(C) in the function definition.
Feb 19 2013
next sibling parent reply "jerro" <a a.com> writes:
 D doesn't use null termination for it's strings, strings are 
 immutable(char)[]. You can form a D slice from a pointer by 
 going
 slice = ptr[0..length]
 where length is the length of the array the pointer represents.
 You can't just take a c style string and expect writeln to work 
 with it.
You can use std.conv.to to convert a C string to a D string.
Feb 19 2013
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 19 February 2013 at 19:06:47 UTC, jerro wrote:
 D doesn't use null termination for it's strings, strings are 
 immutable(char)[]. You can form a D slice from a pointer by 
 going
 slice = ptr[0..length]
 where length is the length of the array the pointer represents.
 You can't just take a c style string and expect writeln to 
 work with it.
You can use std.conv.to to convert a C string to a D string.
Good to know. If you do know the length beforehand though, slicing will O(1) as opposed to O(n)
Feb 19 2013
prev sibling parent reply "Chris" <wendlec tcd.ie> writes:
On Tuesday, 19 February 2013 at 17:40:03 UTC, John Colvin wrote:
 D doesn't use null termination for it's strings, strings are 
 immutable(char)[]. You can form a D slice from a pointer by 
 going
 slice = ptr[0..length]
 where length is the length of the array the pointer represents.
 You can't just take a c style string and expect writeln to work 
 with it.
I had tried to!string() and the slicing already but to no avail. I think it has something to do with pointer and the stack. Here is what I get export void printThis(ref char[] str) { printf("Incoming printf: %s\n", str); writeln(str); writeln(str.ptr); writeln(str[0..5]); writeln(to!string(str.ptr)); writeln(to!string(str.ptr[0..5])); writeln(to!string(str)); // Error! } OUTPUT: Incoming printf: Hello world ☺ 27FBCC ☺ ☺ ☺ Traceback (most recent call last): File "loaddll.py", line 6, in <module> lib.printThis(c_char_p("Hello world")) WindowsError: exception: access violation reading 0x00284000 If I use writefln("%s\n", str) I get the full garbage which starts with ☺ ☺ 0³' ░ï^ ♦ ☺ (←5 8¹' l¹' x ' x ' 1þ→↔A±ôj ░³' .ö→↔☺◄ D0 ► ô→↔ æ"☻­C←↔Jô→↔È1←↔ ß"☻¶)"☻ °;5 +Æ→↔á╚h P®h P®h `³' ► P³' 0³' ░ï^ p ³' ☺ a↨»wxY#☻êS→↔D0 ►P®h ☺◄ ░<$☻ á╚h ►Q→↔á╚h xY#☻ ░<$☻ D0 ► \¬▲ P®h ─²' ¬Ø♫▲á╚h P®h á╚h ☺ ☺ P®h EÑ♫▲á╚h ☺ ☺ æ"☻╚Ui Ó("☻hj] └Ui ╣¢♫▲Ô╚♫▲─²' ­½] ­½] └Ui  ' æ"☻Ó("☻ fÉ"☻☺ 4É"☻ ÞJh (←5 á*♥▲­½] └Ui H▒ ▲─Ui ♥          p■] ıÏ♫▲êTi ­½] ú/5  ' (Ô$☻+¯♫▲ê♀"☻ú/5 └Ui (←5 2┘♫▲­½] êTi ­&^ 3◄▲­½] ­&^ ­&^ (Ô$☻Ó┌5 ED◄▲(Ô$☻­&^ ­&^ ☺ ú/5ttt®/5 ¤I◄tttú/5 ☺☺ ­&^ ­&^ ☺  ' ☺ tttú/5 ▬☻nt­&^ ☺ ¨Rtttú/5 ☺  ' ðÚmttt§ç♥tttú/5 ☺  ' |3 ↔☺ ê ' ú/5 ttt Âü☺ !☻ +e▲Q ╚¬$Q +e ▲Q m◄ ↔☻ É/5 ÿ 5 í9┐w Ó²~ X ' ×⌂P¢─ ' U▬ ↔±þÿj ö ' ¬3╚v Ó ²~È ' ‗×┐w Ó²~M∟ºw Ó²~ á '     ıq├w®&> ý ' ┼×┐w¶‼ ↔ Ó ²~ ¶‼ ↔ Ó²~ Actx ☺ ♀3 ▄ ¶ ☺ 4 |☺ ☺ ☻ N´&→ÿ☻ D Ó☻ `☻ ║q2¾ ♣ J î♣ ▲♥ [IY 2 ♥ ═Û╬2Ó♂ B $♀ 6♥ ╚_P8\☼ ^ ╝☼ h♥ D♣(▒$‼ V |‼ ÿ♥ ► ý ☻ ☺ |☺ ð§ ☺ ☻ L↨ á ☺ ♥ ý▲ î♫ ☺ ♦ x- ¶♥ ☻ ♣ î0 ÿ ☻ ♠ $1 ╠ ☻ ­1 ­ ☺ Ó2 ( ☻ ♂ 3 ♦ ☺ SsHd, ☺ ☺ ☺ ♠ î ☺ ÿ§ , ^ ^ ☻ $ 8 C : \ W i n d o w s \ W i n S x s \ N´&→∟☺ D d☺ `☻ ☺ ║q2¾─♥ J ►♦ ▲♥ ☻ [IY-0 2 d ♥ ♥ ═Û╬2d B ¿ ^ ♫ h♥ ♣ D♣(▒¿◄ V ↕ ÿ♥ ♠ M i c r o s o f t . W i n d o w s . S y s t e m C o m p a t i b l e l ☺ ♀☺ ð☺ ☻ , ▄☻ º¹Lí$♦╩☺☺ ETC. ETC......
 Also, I think you should have extern(C) in the function 
 definition.
I based the DLL on this how-to: http://dlang.org/dll.html
Feb 20 2013
parent reply "Chris" <wendlec tcd.ie> writes:
I tried extern (C) which has some advantages. However, it still 
doesn't produce the desired result (tried slicing too).

extern(C) {
   export void synthesize(ref char[] str) {
	printf("Incoming printf: %s\n", &str);
	writefln("writefln %s", &str);
	writefln("writefln %s", to!string(&str));
	writefln("writefln %s", to!string(str));
   }
}

OUTPUT:

Incoming printf: Hello from Python
writefln 2366A34
writefln 2366A34
Traceback (most recent call last):
   File "loaddll.py", line 4, in <module>
     lib.printThis(c_char_p("Hello from Python"))
WindowsError: [Error -532414463] Windows Error 0xE0440001
Feb 20 2013
parent reply "Mike Parker" <aldacron gmail.com> writes:
On Wednesday, 20 February 2013 at 12:48:53 UTC, Chris wrote:
 I tried extern (C) which has some advantages. However, it still 
 doesn't produce the desired result (tried slicing too).

 extern(C) {
   export void synthesize(ref char[] str) {
 	printf("Incoming printf: %s\n", &str);
 	writefln("writefln %s", &str);
 	writefln("writefln %s", to!string(&str));
 	writefln("writefln %s", to!string(str));
   }
 }

 OUTPUT:

 Incoming printf: Hello from Python
 writefln 2366A34
 writefln 2366A34
 Traceback (most recent call last):
   File "loaddll.py", line 4, in <module>
     lib.printThis(c_char_p("Hello from Python"))
 WindowsError: [Error -532414463] Windows Error 0xE0440001
Your function is being called from Python, correct? Then in addition to the extern(C), the argument needs to be a char*, not a D array or a reference to one.
Feb 20 2013
parent reply "Chris" <wendlec tcd.ie> writes:
On Wednesday, 20 February 2013 at 14:05:40 UTC, Mike Parker wrote:
 Your function is being called from Python, correct? Then in 
 addition to the extern(C), the argument needs to be a char*, 
 not a D array or a reference to one.
Correct, and it works _now_*! The lines printf("Incoming printf: %s\n", str); writefln("writefln %s", to!string(str)); Print now: Incoming printf: Hello from Python writefln Hello from Python So the correct signature is extern (C) {export void printThis(char* str);} Thanks you guys! I'm so glad I don't have to write a C-wrapper! *("_now_" because I tried char* with extern (C) before as I would in my other Python modules, but it didn't work for some reason. Must have overlooked something. Mea culpa!).
Feb 20 2013
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Chris:

 extern (C) {export void printThis(char* str);}
Maybe the D wiki should contain info to save some time and experiments to people. Possible alternative syntax: extern(C) export void printThis(char* str); Also, think if you want some const: extern(C) export void printThis(const(char)* str); Bye, bearophile
Feb 20 2013
parent "Chris" <wendlec tcd.ie> writes:
On Wednesday, 20 February 2013 at 14:36:56 UTC, bearophile wrote:
 Chris:

 extern (C) {export void printThis(char* str);}
Maybe the D wiki should contain info to save some time and experiments to people.
I agree and I am glad that the people on this forum are always willing to help. I will soon be able to write a book about interfacing to D (via C) from various languages, ha ha!
 Possible alternative syntax:

 extern(C) export void printThis(char* str);

 Also, think if you want some const:

 extern(C) export void printThis(const(char)* str);
Yep, const is next on my list. By the way, Python could call the functions in the DLL _without_ extern (C). The name would be mangled though, e.g.: lib.D8abairDLL10printThisFxPaZv(...) It wasn't until arguments were passed to D that the problems kicked in.
Feb 20 2013
prev sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 20 February 2013 at 14:28:06 UTC, Chris wrote:
 On Wednesday, 20 February 2013 at 14:05:40 UTC, Mike Parker 
 wrote:
 Your function is being called from Python, correct? Then in 
 addition to the extern(C), the argument needs to be a char*, 
 not a D array or a reference to one.
Correct, and it works _now_*! The lines printf("Incoming printf: %s\n", str); writefln("writefln %s", to!string(str)); Print now: Incoming printf: Hello from Python writefln Hello from Python So the correct signature is extern (C) {export void printThis(char* str);} Thanks you guys! I'm so glad I don't have to write a C-wrapper! *("_now_" because I tried char* with extern (C) before as I would in my other Python modules, but it didn't work for some reason. Must have overlooked something. Mea culpa!).
yeah, although extern(C) doesn't have to have any braces. I've never had to use export before, but then that could be a .so/.dylib vs .dll thing
Feb 20 2013
parent "Chris" <wendlec tcd.ie> writes:
On Wednesday, 20 February 2013 at 14:43:06 UTC, John Colvin wrote:
 On Wednesday, 20 February 2013 at 14:28:06 UTC, Chris wrote:
 On Wednesday, 20 February 2013 at 14:05:40 UTC, Mike Parker 
 wrote:
 Your function is being called from Python, correct? Then in 
 addition to the extern(C), the argument needs to be a char*, 
 not a D array or a reference to one.
Correct, and it works _now_*! The lines printf("Incoming printf: %s\n", str); writefln("writefln %s", to!string(str)); Print now: Incoming printf: Hello from Python writefln Hello from Python So the correct signature is extern (C) {export void printThis(char* str);} Thanks you guys! I'm so glad I don't have to write a C-wrapper! *("_now_" because I tried char* with extern (C) before as I would in my other Python modules, but it didn't work for some reason. Must have overlooked something. Mea culpa!).
yeah, although extern(C) doesn't have to have any braces.
You are right of course!
 I've never had to use export before, but then that could be a 
 .so/.dylib vs .dll thing
"export" is used in the DLL how-to. I never had to use it for my .so/.dylib modules either. I am new to Windows and it is a very strange beast. Like cooking without pots and pans.
Feb 20 2013