www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Calling external programs from D

reply Tydr Schnubbis <fake address.dude> writes:
I'm making a gui app with DWT, and I need a way to run an external, 
command line tool and get its output.  Either directly from the tool's 
stdout, or by having it write to a file first.

I'm working on windows.  The system() function is unusable because it 
opens a new command line window when it starts the tool.  That you can 
get it to close the window again really fast by using 'start' is not 
good enough

It doesn't have to be a portable way.  If someone can tell me how to 
call _popen or something in msvcrt.dll, I would be happy.
Apr 05 2006
parent reply kris <foo bar.com> writes:
Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external, 
 command line tool and get its output.  Either directly from the tool's 
 stdout, or by having it write to a file first.
 
 I'm working on windows.  The system() function is unusable because it 
 opens a new command line window when it starts the tool.  That you can 
 get it to close the window again really fast by using 'start' is not 
 good enough
 
 It doesn't have to be a portable way.  If someone can tell me how to 
 call _popen or something in msvcrt.dll, I would be happy.
I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan
Apr 05 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,  
 command line tool and get its output.  Either directly from the tool's  
 stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because it  
 opens a new command line window when it starts the tool.  That you can  
 get it to close the window again really fast by using 'start' is not  
 good enough
  It doesn't have to be a portable way.  If someone can tell me how to  
 call _popen or something in msvcrt.dll, I would be happy.
I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan
Here they/it is :) Regan
Apr 05 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Regan Heath wrote:
 On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,  
 command line tool and get its output.  Either directly from the tool's  
 stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because it  
 opens a new command line window when it starts the tool.  That you can  
 get it to close the window again really fast by using 'start' is not  
 good enough
  It doesn't have to be a portable way.  If someone can tell me how to  
 call _popen or something in msvcrt.dll, I would be happy.
I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan
Here they/it is :) Regan
Thanks! But it doesn't work for me. Seems that it blocks the new process from acessing the network. Any ideas what to do?
Apr 05 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,  
 command line tool and get its output.  Either directly from the tool's  
 stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because it  
 opens a new command line window when it starts the tool.  That you can  
 get it to close the window again really fast by using 'start' is not  
 good enough
  It doesn't have to be a portable way.  If someone can tell me how to  
 call _popen or something in msvcrt.dll, I would be happy.
I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan
Here they/it is :) Regan
Thanks! But it doesn't work for me. Seems that it blocks the new process from acessing the network. Any ideas what to do?
Here's a minimal test to show the problem: import lib.process; import std.stdio; void main() { Process proc; proc = new Process("ping google.com"); writef(proc.readLine()); } Compile: "dmd test.d lib/process.d lib/pipestream.d" Output: "Ping request could not find host google.com. Please check the name and try again." ;)
Apr 05 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 05 Apr 2006 19:22:25 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
 On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,   
 command line tool and get its output.  Either directly from the  
 tool's  stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because  
 it  opens a new command line window when it starts the tool.  That  
 you can  get it to close the window again really fast by using  
 'start' is not  good enough
  It doesn't have to be a portable way.  If someone can tell me how  
 to  call _popen or something in msvcrt.dll, I would be happy.
I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan
Here they/it is :) Regan
Thanks! But it doesn't work for me. Seems that it blocks the new process from acessing the network. Any ideas what to do?
Here's a minimal test to show the problem: import lib.process; import std.stdio; void main() { Process proc; proc = new Process("ping google.com"); writef(proc.readLine()); } Compile: "dmd test.d lib/process.d lib/pipestream.d"
 Output: "Ping request could not find host google.com. Please check the  
 name and try again."
(Cross posted to digitalmars.D so more people see it) (You have to move the readLine function from private to public in pipestream, that was obviously a mistake on my part) I get the same result. Odd. I thought at first it might be because ping uses enviroment variables to find/use the DNS service, but adding the current enviroment variables to the new process makes no difference: import lib.process; import std.stdio; import std.string; import std.c.string; extern(C) extern char **_environ; void main() { Process proc; proc = new Process(); for(int i = 0; _environ[i]; i++) { proc.addEnv(toString(_environ[i]).dup); } proc.execute("ping www.google.com"); writef(proc.readLine()); } I also tried using CreateProcessAsUser with the handle returned by: OpenProcessToken(GetCurrentProcess(), ..etc.. it made no difference (not that I expected it to, but you never know). Does anyone have any idea why ping cannot access the DNS service? Regan
Apr 05 2006
parent reply Stuart Delaney <Stuart_member pathlink.com> writes:
There's a bug in the makeBlock functions in process.d. The first parameter to
calloc should be 1 not 0. With that change (and the private readLine one) it
works fine for me.  Don't have an answer to the OP's DNS problem though.

hope this helps. cheers,
Stu

In article <ops7j4wey023k2f5 nrage.netwin.co.nz>, Regan Heath says...
On Wed, 05 Apr 2006 19:22:25 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
 On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,   
 command line tool and get its output.  Either directly from the  
 tool's  stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because  
 it  opens a new command line window when it starts the tool.  That  
 you can  get it to close the window again really fast by using  
 'start' is not  good enough
  It doesn't have to be a portable way.  If someone can tell me how  
 to  call _popen or something in msvcrt.dll, I would be happy.
I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan
Here they/it is :) Regan
Thanks! But it doesn't work for me. Seems that it blocks the new process from acessing the network. Any ideas what to do?
Here's a minimal test to show the problem: import lib.process; import std.stdio; void main() { Process proc; proc = new Process("ping google.com"); writef(proc.readLine()); } Compile: "dmd test.d lib/process.d lib/pipestream.d"
 Output: "Ping request could not find host google.com. Please check the  
 name and try again."
(Cross posted to digitalmars.D so more people see it) (You have to move the readLine function from private to public in pipestream, that was obviously a mistake on my part) I get the same result. Odd. I thought at first it might be because ping uses enviroment variables to find/use the DNS service, but adding the current enviroment variables to the new process makes no difference: import lib.process; import std.stdio; import std.string; import std.c.string; extern(C) extern char **_environ; void main() { Process proc; proc = new Process(); for(int i = 0; _environ[i]; i++) { proc.addEnv(toString(_environ[i]).dup); } proc.execute("ping www.google.com"); writef(proc.readLine()); } I also tried using CreateProcessAsUser with the handle returned by: OpenProcessToken(GetCurrentProcess(), ..etc.. it made no difference (not that I expected it to, but you never know). Does anyone have any idea why ping cannot access the DNS service? Regan
Apr 06 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney  
<Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first  
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine  
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem though.
You're dead right. With those changes it works for me too. Regan
Apr 06 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney  
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first  
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine  
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem though.
You're dead right. With those changes it works for me too.
I've fixed the calloc calls too, but it doesn't help. If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.
Apr 06 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney  
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first  
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine  
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem though.
You're dead right. With those changes it works for me too.
I've fixed the calloc calls too, but it doesn't help. If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.
Could it be that CreateProcessA is used wrong somehow? I don't know the win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network? Where would I start looking if I want to fix this? The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'. But it seems to be the same function. Here's what the call looks like: CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)
Apr 18 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
 Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney   
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first   
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine   
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem  
 though.
You're dead right. With those changes it works for me too.
I've fixed the calloc calls too, but it doesn't help. If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.
Could it be that CreateProcessA is used wrong somehow? I don't know the win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network? Where would I start looking if I want to fix this? The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'. But it seems to be the same function. Here's what the call looks like: CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)
CreateProcessA is the ascii version. CreateProcessW is the unicode (UTF-16) version. std.string.toStringz converts the UTF-8 char[] into ascii std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if you want to call CreateProcessW instead. The common cause of the "4invalid UTF-8 sequence" error is trying to output non-ascii characters to the windows console. Can you post your current code here. Regan
Apr 18 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Regan Heath wrote:
 On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake address.dude>  
 wrote:
 Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney   
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first   
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine   
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem  
 though.
You're dead right. With those changes it works for me too.
I've fixed the calloc calls too, but it doesn't help. If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.
Could it be that CreateProcessA is used wrong somehow? I don't know the win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network? Where would I start looking if I want to fix this? The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'. But it seems to be the same function. Here's what the call looks like: CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)
CreateProcessA is the ascii version. CreateProcessW is the unicode (UTF-16) version. std.string.toStringz converts the UTF-8 char[] into ascii std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if you want to call CreateProcessW instead. The common cause of the "4invalid UTF-8 sequence" error is trying to output non-ascii characters to the windows console. Can you post your current code here.
Sure. As you can see it's full of weird characters... import lib.process; import std.stdio; void main() { Process proc; // ping my router proc = new Process("ping 192.168.0.1"); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); } I have only made readLine public, and fixed the four calloc calls, no other changes have been made to your files. Compile with: dmd test.d lib/process.d lib/pipestream.d This prints: Pinging Error: 4invalid UTF-8 sequence If I ping google.com instead, I get this: Ping request could not find host google.com. Please check the name and try again . Error: ReadFile: The pipe has been ended. --------- Does both of these work for you? I have no idea what would cause any of these problems. I have winxp SP2 US. CreateProcessA is in kernel32.dll, of which I have version 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158). dmd version 0.154, got the same results with 0.148. :/
Apr 19 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 19 Apr 2006 16:53:18 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
 Regan Heath wrote:
 On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake address.dude>   
 wrote:
 Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney    
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first    
 parameter to
 calloc should be 1 not 0. With that change (and the private  
 readLine   one) it
 works fine for me.  Don't have an answer to the OP's DNS problem   
 though.
You're dead right. With those changes it works for me too.
I've fixed the calloc calls too, but it doesn't help. If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.
Could it be that CreateProcessA is used wrong somehow? I don't know the win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network? Where would I start looking if I want to fix this? The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'. But it seems to be the same function. Here's what the call looks like: CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)
CreateProcessA is the ascii version. CreateProcessW is the unicode (UTF-16) version. std.string.toStringz converts the UTF-8 char[] into ascii std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if you want to call CreateProcessW instead. The common cause of the "4invalid UTF-8 sequence" error is trying to output non-ascii characters to the windows console. Can you post your current code here.
Sure. As you can see it's full of weird characters... import lib.process; import std.stdio; void main() { Process proc; // ping my router proc = new Process("ping 192.168.0.1"); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); } I have only made readLine public, and fixed the four calloc calls, no other changes have been made to your files. Compile with: dmd test.d lib/process.d lib/pipestream.d This prints: Pinging Error: 4invalid UTF-8 sequence If I ping google.com instead, I get this: Ping request could not find host google.com. Please check the name and try again . Error: ReadFile: The pipe has been ended. --------- Does both of these work for you? I have no idea what would cause any of these problems. I have winxp SP2 US. CreateProcessA is in kernel32.dll, of which I have version 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158). dmd version 0.154, got the same results with 0.148.
It's quite odd, try this: import lib.process; import std.stdio; import std.string; import std.c.string; extern(C) extern char **_environ; void main() { Process proc; proc = new Process(); for(int i = 0; _environ[i]; i++) { proc.addEnv(toString(_environ[i]).dup); } //proc.execute("ping www.google.com"); proc.execute("ping 192.168.0.1"); while(true) writefln("%s",proc.readLine()); } without the addEnv calls above I get the behaviour you're describing. With them it works. Without them, and using printf I can see that ping responds with: "Pinging with 32 bytes of data:" note the weird characters there. At first I thought maybe the command line I was passing to CreateProcessA was temporary and being collected by the GC, so I changed process.d to use: cmd = strdup(std.string.toStringz(command)); where cmd is a member of Process - so will persist as long as it does. That made no difference. I have no idea why it's doing that, perhaps it reads it's args in a strange way?? I might write a debug program and run that passing different args etc to see if I can replicate the odd behaviour and figure out where it comes from. Regan
Apr 19 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Regan Heath wrote:
  The common cause of the "4invalid UTF-8 sequence" error is trying to   
 output non-ascii characters to the windows console. Can you post your   
 current code here.
Sure. As you can see it's full of weird characters... import lib.process; import std.stdio; void main() { Process proc; // ping my router proc = new Process("ping 192.168.0.1"); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); } I have only made readLine public, and fixed the four calloc calls, no other changes have been made to your files. Compile with: dmd test.d lib/process.d lib/pipestream.d This prints: Pinging Error: 4invalid UTF-8 sequence If I ping google.com instead, I get this: Ping request could not find host google.com. Please check the name and try again . Error: ReadFile: The pipe has been ended. --------- Does both of these work for you? I have no idea what would cause any of these problems. I have winxp SP2 US. CreateProcessA is in kernel32.dll, of which I have version 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158). dmd version 0.154, got the same results with 0.148.
It's quite odd, try this: import lib.process; import std.stdio; import std.string; import std.c.string; extern(C) extern char **_environ; void main() { Process proc; proc = new Process(); for(int i = 0; _environ[i]; i++) { proc.addEnv(toString(_environ[i]).dup); } //proc.execute("ping www.google.com"); proc.execute("ping 192.168.0.1"); while(true) writefln("%s",proc.readLine()); } without the addEnv calls above I get the behaviour you're describing. With them it works.
Works for me too, thanks!
 Without them, and using printf I can see that ping responds with:
 
 "Pinging  with 32 bytes of data:"
 
 note the weird characters there. At first I thought maybe the command line  
 I was passing to CreateProcessA was temporary and being collected by the  
 GC, so I changed process.d to use:
 
 cmd = strdup(std.string.toStringz(command));
 
 where cmd is a member of Process - so will persist as long as it does.  
 That made no difference. I have no idea why it's doing that, perhaps it  
 reads it's args in a strange way?? I might write a debug program and run  
 that passing different args etc to see if I can replicate the odd  
 behaviour and figure out where it comes from.
Not sure if this helps: http://www.digitalmars.com/techtips/windows_utf.html
Apr 19 2006
parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 20 Apr 2006 00:48:55 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
  without the addEnv calls above I get the behaviour you're describing.  
 With  them it works.
Works for me too, thanks!
NP.
 Without them, and using printf I can see that ping responds with:
  "Pinging  with 32 bytes of data:"
  note the weird characters there. At first I thought maybe the command  
 line  I was passing to CreateProcessA was temporary and being collected  
 by the  GC, so I changed process.d to use:
  cmd = strdup(std.string.toStringz(command));
  where cmd is a member of Process - so will persist as long as it  
 does.  That made no difference. I have no idea why it's doing that,  
 perhaps it  reads it's args in a strange way?? I might write a debug  
 program and run  that passing different args etc to see if I can  
 replicate the odd  behaviour and figure out where it comes from.
Not sure if this helps: http://www.digitalmars.com/techtips/windows_utf.html
Nope :( It's my understanding that if you're using ASCII, as we are, you can call the A functions without any conversion, you simply need to ensure there is a null terminater on the string (which is what toStringz does). In any case I tried both toMBSz with CreateProcessA and toUTF16 with CreateProcessW, it made no difference. Those results plus the same ones I got dup'ing the command value suggest to me that the command we're passing isn't the problem. Further, the complete output from ping later shows the parameter correctly, see: "Pinging with 32 bytes of data: Reply from 192.168.1.1: bytes=32 time=5ms TTL=255" (which also means ping can operate without enviroment vars provided it does not need to do a DNS lookup on the name you give it, in this case an ip) I think there may be a bug in ping .. as much as I hate to suggest it (because in most cases you later find out you're wrong). I suspect it somehow uses an enviroment variable without error checking when printing that first line, resulting in garbage being printed. Regan
Apr 19 2006