www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Calling external programs from D

reply Tydr Schnubbis <fake address.dude> writes:
I'm making a gui app with DWT, and I need a way to run an external, 
command line tool and get its output.  Either directly from the tool's 
stdout, or by having it write to a file first.

I'm working on windows.  The system() function is unusable because it 
opens a new command line window when it starts the tool.  That you can 
get it to close the window again really fast by using 'start' is not 
good enough

It doesn't have to be a portable way.  If someone can tell me how to 
call _popen or something in msvcrt.dll, I would be happy.
Apr 05 2006
parent reply kris <foo bar.com> writes:
Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external, 
 command line tool and get its output.  Either directly from the tool's 
 stdout, or by having it write to a file first.
 
 I'm working on windows.  The system() function is unusable because it 
 opens a new command line window when it starts the tool.  That you can 
 get it to close the window again really fast by using 'start' is not 
 good enough
 
 It doesn't have to be a portable way.  If someone can tell me how to 
 call _popen or something in msvcrt.dll, I would be happy.

I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan
Apr 05 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
------------CYOrmsTJMqxCFBdwjNLfOY
Content-Type: text/plain; format=flowed; delsp=yes; charset=iso-8859-15
Content-Transfer-Encoding: 8bit

On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,  
 command line tool and get its output.  Either directly from the tool's  
 stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because it  
 opens a new command line window when it starts the tool.  That you can  
 get it to close the window again really fast by using 'start' is not  
 good enough
  It doesn't have to be a portable way.  If someone can tell me how to  
 call _popen or something in msvcrt.dll, I would be happy.

I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan

Here they/it is :) Regan ------------CYOrmsTJMqxCFBdwjNLfOY Content-Disposition: attachment; filename=pipestream.d Content-Type: application/octet-stream; name=pipestream.d Content-Transfer-Encoding: 8bit /* * Copyright (c) 2005 * Regan Heath * * Permission to use, copy, modify, distribute and sell this software * and its documentation for any purpose is hereby granted without fee, * provided that the above copyright notice appear in all copies and * that both that copyright notice and this permission notice appear * in supporting documentation. Author makes no representations about * the suitability of this software for any purpose. It is provided * "as is" without express or implied warranty. */ module lib.pipestream; import std.stream; extern(C) char* strdup(char*); version(Windows) { import std.c.windows.windows; import std.windows.syserror; extern(Windows) { alias HANDLE* PHANDLE; BOOL CreatePipe( PHANDLE hReadPipe, PHANDLE hWritePipe, LPSECURITY_ATTRIBUTES lpPipeAttributes, DWORD nSize ); BOOL PeekNamedPipe( HANDLE hNamedPipe, LPVOID lpBuffer, DWORD nBufferSize, LPDWORD lpBytesRead, LPDWORD lpTotalBytesAvail, LPDWORD lpBytesLeftThisMessage ); } } version(linux) { private import std.c.stdlib; extern (C) char* strerror(int); } version(Windows) { class PipeException : Exception { this(char[] msg) { super(msg ~ ": " ~ sysErrorString(GetLastError())); } } class PipeStream : Stream { this(uint bufferSize = 0) { SECURITY_ATTRIBUTES security; security.nLength = security.sizeof; security.lpSecurityDescriptor = null; security.bInheritHandle = true; if (!CreatePipe(&read,&write,&security,bufferSize)) throw new PipeException("CreatePipe"); writeable = true; readable = true; seekable = false; isopen = true; } HANDLE readHandle() { return read; } HANDLE writeHandle() { return write; } void closeRead() { CloseHandle(readHandle); read = INVALID_HANDLE_VALUE; readable = false; if (!writeable) isopen = false; } void closeWrite() { CloseHandle(writeHandle); write = INVALID_HANDLE_VALUE; writeable = false; if (!readable) isopen = false; } override void close() { closeRead(); closeWrite(); } override ulong seek(long offset, SeekPos whence) { assertSeekable(); } override size_t readBlock(void* buffer, size_t size) { size_t bytes = 0; assertReadable(); if (!ReadFile(readHandle,buffer,size,&bytes,null)) throw new PipeException("ReadFile"); return bytes; } override size_t available() { size_t bytes = 0; assertReadable(); if (!PeekNamedPipe(readHandle,null,0,null,&bytes,null)) throw new PipeException("PeekNamedPipe"); return bytes; } override size_t writeBlock(void* buffer, size_t size) { size_t bytes = 0; assertWriteable(); if (!WriteFile(writeHandle,buffer,size,&bytes,null)) throw new PipeException("WriteFile"); return bytes; } override void flush() { assertWriteable(); FlushFileBuffers(writeHandle); } private: HANDLE write = INVALID_HANDLE_VALUE; HANDLE read = INVALID_HANDLE_VALUE; } } version(linux) { class PipeException : Exception { //for some reason getErrno does not link for me? this(char[] msg) { super(msg ~ ": " ~ std.string.toString(strerror(getErrno()))); } } class PipeStream : Stream { this(uint dummy = 0) { if (pipe(handle) == -1) throw new PipeException("pipe(handle)"); writeable = true; readable = true; seekable = false; isopen = true; } int readHandle() { return handle[0]; } int writeHandle() { return handle[1]; } void closeRead() { close(readHandle); readable = false; if (!writeable) isopen = false; } void closeWrite() { close(writeHandle); writeable = false; if (!readable) isopen = false; } override void close() { closeRead(); closeWrite(); } override ulong seek(long offset, SeekPos whence) { assertSeekable(); } override size_t readBlock(void* buffer, size_t size) { size_t bytes; assertReadable(); bytes = read(readHandle,buffer,size); if (bytes == -1) throw new PipeException("read(handle[0])"); return bytes; } override size_t available() { size_t bytes; assertReadable(); if (ioctl(readHandle,FIONREAD,&bytes) == -1) throw new PipeException("ioctl(handle[0])"); return bytes; } override size_t writeBlock(void* buffer, size_t size) { size_t bytes; assertWriteable(); bytes = write(writeHandle,buffer,size); if (bytes == -1) throw new PipeException("write(handle[1])"); return bytes; } override void flush() { assertWriteable(); //writeHandle } private: int handle[2]; } } /+ class Process : Stream { alias std.stdio.writefln debugf; this() { super(); seekable = false; readable = true; writeable = true; isopen = false; } this(char[] command) { this(); open(command); } ~this() { close(); } void addEnv(char[] label, char[] value) { addEnv(label~"="~value); } void addEnv(char[] value) { enviroment ~= value; } void open(char[] command) { if (isopen) close(); startProcess(command); } override void close() { if (!isopen) return ; flush(); stopProcess(0); isopen = false; } override ulong seek(long offset, SeekPos whence) { assertSeekable(); } version(Windows) { override size_t readBlock(void* buffer, size_t size) { size_t bytes = 0; if (!isopen) return 0; if (!ReadFile(output,buffer,size,&bytes,null)) throw new ProcessException("ReadFile"); return bytes; } override size_t writeBlock(void* buffer, size_t size) { size_t bytes = 0; if (!isopen) return 0; if (!WriteFile(input,buffer,size,&bytes,null)) throw new ProcessException("WriteFile"); return bytes; } override size_t available() { size_t bytes = 0; if (!isopen) return 0; if (!PeekNamedPipe(output,null,0,null,&bytes,null)) throw new ProcessException("PeekNamedPipe"); return bytes; } override void flush() { if (!isopen) return ; FlushFileBuffers(output); } } version(linux) { override size_t readBlock(void* buffer, size_t size) { size_t bytes; bytes = read(output,buffer,size); if (bytes == -1) throw new ProcessException("read"); return bytes; } override size_t writeBlock(void* buffer, size_t size) { size_t bytes; bytes = write(output,buffer,size); if (bytes == -1) throw new ProcessException("write"); return bytes; } override size_t available() { size_t bytes; if (ioctl(output,FIONREAD,&bytes) == -1) throw new ProcessException("ioctl"); return bytes; } } private: char[][] enviroment = null; version(Windows) { HANDLE output = INVALID_HANDLE_VALUE; HANDLE input = INVALID_HANDLE_VALUE; PROCESS_INFORMATION* info = null; int bufferSize = 0; char* makeBlock(char[][] from) { char* result = null; uint length = 0; uint upto = 0; foreach(char[] s; from) length += s.length; //total length of strings length += from.length; //add space for a \0 for each string length++; //add space for final terminating \0 result = cast(char*)calloc(0,length); foreach(char[] s; from) { result[upto..upto+s.length] = s[0..s.length]; upto += s.length+1; } return result; } void freeBlock(char* data) { free(data); } void startProcess(char[] command) { HANDLE read1,read2,read3,write1,write2,write3; SECURITY_ATTRIBUTES security; STARTUPINFOA startup; char* env; read1 = read2 = read3 = write1 = write2 = write2 = INVALID_HANDLE_VALUE; security.nLength = security.sizeof; security.lpSecurityDescriptor = null; security.bInheritHandle = true; env = null; try { if (!CreatePipe(&read1,&write1,&security,bufferSize)) throw new ProcessException("CreatePipe"); if (!CreatePipe(&read2,&write2,&security,bufferSize)) throw new ProcessException("CreatePipe"); if (!CreatePipe(&read3,&write3,&security,bufferSize)) throw new ProcessException("CreatePipe"); GetStartupInfoA(&startup); startup.hStdInput = read1; startup.hStdOutput = write2; startup.hStdError = write3; startup.dwFlags = STARTF_USESTDHANDLES; info = new PROCESS_INFORMATION(); env = makeBlock(enviroment); if (!CreateProcessA(null,toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)) throw new ProcessException("CreateProcess"); input = write1; output = read2; isopen = true; } finally { if (!isopen && read2 != INVALID_HANDLE_VALUE) CloseHandle(read2); if (!isopen && write1 != INVALID_HANDLE_VALUE) CloseHandle(write1); if (read1 != INVALID_HANDLE_VALUE) CloseHandle(read1); if (read3 != INVALID_HANDLE_VALUE) CloseHandle(read3); if (write2 != INVALID_HANDLE_VALUE) CloseHandle(write2); if (write3 != INVALID_HANDLE_VALUE) CloseHandle(write3); if (info) CloseHandle(info.hThread); if (env) freeBlock(env); } } void stopProcess(uint exitCode) { if (!TerminateProcess(info.hProcess,exitCode)) throw new ProcessException("TerminateProcess"); isopen = false; CloseHandle(info.hProcess); delete info; info = null; CloseHandle(output); output = INVALID_HANDLE_VALUE; CloseHandle(input); input = INVALID_HANDLE_VALUE; } } version(linux) { int output = 0; int input = 0; int pid = 0; char** makeBlock(char[][] from) { char** result = null; result = cast(char**)calloc(0,(enviroment.length+1) * typeid(char*).sizeof); foreach(uint i, char[] s; from) result[i] = strdup(toStringz(s)); return result; } void freeBlock(char** block) { for(uint i = 0; block[i]; i++) free(block[i]); free(block); } char[][] splitArgs(char[] string, char[] delims) { char[] delims = " \t\r\n"; char[][] results = null; bool isquot = false; int start = -1; for(int i = 0; i < string.length; i++) { if (string[i] == '\"') isquot = !isquot; if (isquot) continue; if (delims.find(string[i]) != -1) { if (start == -1) continue; results ~= string[start..i]; start = -1; continue; } if (start == -1) start = i; } return results; } void startProcess(char[] command) { int xwrite[2]; int xread[2]; try { if (pipe(xwrite) == -1) throw new ProcessException("pipe(xwrite)"); if (pipe(xread) == -1) throw new ProcessException("pipe(xread)"); if (fcntl(input, F_SETFD, 1) == -1) throw new ProcessException("fcntl(input)"); if (fcntl(output, F_SETFD, 1) == -1) throw new ProcessException("fcntl(output)"); if (fcntl(fileno(stdin), F_SETFD, 1) == -1) throw new ProcessException("fcntl(stdin)"); if (fcntl(fileno(stdout), F_SETFD, 1) == -1) throw new ProcessException("fcntl(stdout)"); if (fcntl(fileno(stderr), F_SETFD, 1) == -1) throw new ProcessException("fcntl(stderr)"); pid = fork(); if (pid == 0) { /* child */ char** args = null; char** env = null; //not sure if we can even throw here? if (dup2(xwrite[1],STDOUT_FILENO) == -1) {} //throw new ProcessException("dup2(xwrite[1])"); if (dup2(xread[0], STDIN_FILENO) == -1) {} //throw new ProcessException("dup2(xread[0])"); close(xwrite[1]); close(xread[0]); /* set child uid/gid here */ //if (setuid(uid) == -1) throw new ProcessException("setuid"); //if (setgid(gid) == -1) throw new ProcessException("setgid"); args = makeBlock(splitArgs(command)); env = makeBlock(enviroment); execve(args[0],args,env); //this does not return on success //can we throw? how to notify parent of failure? exit(1); } /* parent */ isopen = true; } finally { close(xwrite[1]); close(xread[0]); } } void stopProcess(uint dummy) { int r; if (pid == 0) return; if (kill(pid, SIGTERM) == -1) throw new ProcessException("kill"); for(uint i = 0; i < 100; i++) { r = waitpid(pid,null,WNOHANG|WUNTRACED); if (r == -1) throw new ProcessException("waitpid"); if (r == pid) break; usleep(50000); } isopen = false; close(output); close(input); pid = 0; } } } +/ ------------CYOrmsTJMqxCFBdwjNLfOY Content-Disposition: attachment; filename=process.d Content-Type: application/octet-stream; name=process.d Content-Transfer-Encoding: 8bit /* * Copyright (c) 2005 * Regan Heath * * Permission to use, copy, modify, distribute and sell this software * and its documentation for any purpose is hereby granted without fee, * provided that the above copyright notice appear in all copies and * that both that copyright notice and this permission notice appear * in supporting documentation. Author makes no representations about * the suitability of this software for any purpose. It is provided * "as is" without express or implied warranty. */ module lib.process; private import std.c.stdlib; import lib.pipestream; extern(C) char* strdup(char*); version(Windows) { import std.c.windows.windows; import std.windows.syserror; extern(Windows) { struct PROCESS_INFORMATION { HANDLE hProcess; HANDLE hThread; DWORD dwProcessId; DWORD dwThreadId; } alias PROCESS_INFORMATION* PPROCESS_INFORMATION, LPPROCESS_INFORMATION; struct STARTUPINFOA { DWORD cb; LPSTR lpReserved; LPSTR lpDesktop; LPSTR lpTitle; DWORD dwX; DWORD dwY; DWORD dwXSize; DWORD dwYSize; DWORD dwXCountChars; DWORD dwYCountChars; DWORD dwFillAttribute; DWORD dwFlags; WORD wShowWindow; WORD cbReserved2; LPBYTE lpReserved2; HANDLE hStdInput; HANDLE hStdOutput; HANDLE hStdError; } alias STARTUPINFOA* LPSTARTUPINFOA; struct STARTUPINFOW { DWORD cb; LPWSTR lpReserved; LPWSTR lpDesktop; LPWSTR lpTitle; DWORD dwX; DWORD dwY; DWORD dwXSize; DWORD dwYSize; DWORD dwXCountChars; DWORD dwYCountChars; DWORD dwFillAttribute; DWORD dwFlags; WORD wShowWindow; WORD cbReserved2; LPBYTE lpReserved2; HANDLE hStdInput; HANDLE hStdOutput; HANDLE hStdError; } alias STARTUPINFOW* LPSTARTUPINFOW; VOID GetStartupInfoA(LPSTARTUPINFOA lpStartupInfo); VOID GetStartupInfoW(LPSTARTUPINFOW lpStartupInfo); uint STARTF_USESHOWWINDOW = 0x00000001; uint STARTF_USESIZE = 0x00000002; uint STARTF_USEPOSITION = 0x00000004; uint STARTF_USECOUNTCHARS = 0x00000008; uint STARTF_USEFILLATTRIBUTE = 0x00000010; uint STARTF_RUNFULLSCREEN = 0x00000020; uint STARTF_FORCEONFEEDBACK = 0x00000040; uint STARTF_FORCEOFFFEEDBACK = 0x00000080; uint STARTF_USESTDHANDLES = 0x00000100; /+#if(WINVER >= 0x0400) #define STARTF_USEHOTKEY 0x00000200 #endif /* WINVER >= 0x0400 */ +/ BOOL CreateProcessA( LPCSTR lpApplicationName, LPSTR lpCommandLine, LPSECURITY_ATTRIBUTES lpProcessAttributes, LPSECURITY_ATTRIBUTES lpThreadAttributes, BOOL bInheritHandles, DWORD dwCreationFlags, LPVOID lpEnvironment, LPCSTR lpCurrentDirectory, LPSTARTUPINFOA lpStartupInfo, LPPROCESS_INFORMATION lpProcessInformation ); BOOL CreateProcessW( LPCWSTR lpApplicationName, LPWSTR lpCommandLine, LPSECURITY_ATTRIBUTES lpProcessAttributes, LPSECURITY_ATTRIBUTES lpThreadAttributes, BOOL bInheritHandles, DWORD dwCreationFlags, LPVOID lpEnvironment, LPCWSTR lpCurrentDirectory, LPSTARTUPINFOW lpStartupInfo, LPPROCESS_INFORMATION lpProcessInformation ); // // dwCreationFlag values // uint DEBUG_PROCESS = 0x00000001; uint DEBUG_ONLY_THIS_PROCESS = 0x00000002; uint CREATE_SUSPENDED = 0x00000004; uint DETACHED_PROCESS = 0x00000008; uint CREATE_NEW_CONSOLE = 0x00000010; uint NORMAL_PRIORITY_CLASS = 0x00000020; uint IDLE_PRIORITY_CLASS = 0x00000040; uint HIGH_PRIORITY_CLASS = 0x00000080; uint REALTIME_PRIORITY_CLASS = 0x00000100; uint CREATE_NEW_PROCESS_GROUP = 0x00000200; uint CREATE_UNICODE_ENVIRONMENT = 0x00000400; uint CREATE_SEPARATE_WOW_VDM = 0x00000800; uint CREATE_SHARED_WOW_VDM = 0x00001000; uint CREATE_FORCEDOS = 0x00002000; uint CREATE_DEFAULT_ERROR_MODE = 0x04000000; uint CREATE_NO_WINDOW = 0x08000000; uint PROFILE_USER = 0x10000000; uint PROFILE_KERNEL = 0x20000000; uint PROFILE_SERVER = 0x40000000; BOOL TerminateProcess(HANDLE hProcess, UINT uExitCode); } } version(linux) { extern (C) char* strerror(int); } class ProcessException : Exception { version(Windows) { this(char[] msg) { super(msg ~ ": " ~ sysErrorString(GetLastError())); } } version(linux) { //for some reason getErrno does not link for me? this(char[] msg) { super(msg ~ ": " ~ std.string.toString(strerror(getErrno()))); } } } class Process { this() { } this(char[] command) { this(); execute(command); } void execute(char[] command) { if (running) kill(); startProcess(command); } void kill() { if (!running) return; stopProcess(0); } void addEnv(char[] label, char[] value) { addEnv(label~"="~value); } void addEnv(char[] value) { enviroment ~= value; } private: char[][] enviroment = null; bool running = false; PipeStream pout = null; PipeStream perr = null; PipeStream pin = null; char[] readLine() { return pout.readLine(); } char[] readError() { return perr.readLine(); } void writeLine(char[] line) { pin.writeLine(line); } version(Windows) { PROCESS_INFORMATION *info = null; char* makeBlock(char[][] from) { char* result = null; uint length = 0; uint upto = 0; foreach(char[] s; from) length += s.length; //total length of strings length += from.length; //add space for a \0 for each string length++; //add space for final terminating \0 result = cast(char*)calloc(0,length); foreach(char[] s; from) { result[upto..upto+s.length] = s[0..s.length]; upto += s.length+1; } return result; } void freeBlock(char* data) { free(data); } void startProcess(char[] command) { STARTUPINFOA startup; char* env = null; try { pout = new PipeStream(); perr = new PipeStream(); pin = new PipeStream(); GetStartupInfoA(&startup); startup.hStdInput = pin.readHandle; startup.hStdOutput = pout.writeHandle; startup.hStdError = perr.writeHandle; startup.dwFlags = STARTF_USESTDHANDLES; info = new PROCESS_INFORMATION(); env = makeBlock(enviroment); if (!CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)) throw new ProcessException("CreateProcess"); running = true; } finally { if (env) freeBlock(env); if (running) { CloseHandle(info.hThread); pin.closeRead(); pout.closeWrite(); perr.closeWrite(); } else { if (info) info = null; pout = null; perr = null; pin = null; } } } void stopProcess(uint exitCode) { if (!TerminateProcess(info.hProcess,exitCode)) throw new ProcessException("TerminateProcess"); running = false; CloseHandle(info.hProcess); info = null; pout = null; perr = null; pin = null; } } version(linux) { int pid; char** makeBlock(char[][] from) { char** result = null; result = cast(char**)calloc(0,(enviroment.length+1) * typeid(char*).sizeof); foreach(uint i, char[] s; from) result[i] = strdup(toStringz(s)); return result; } void freeBlock(char** block) { for(uint i = 0; block[i]; i++) free(block[i]); free(block); } char[][] splitArgs(char[] string, char[] delims) { char[] delims = " \t\r\n"; char[][] results = null; bool isquot = false; int start = -1; for(int i = 0; i < string.length; i++) { if (string[i] == '\"') isquot = !isquot; if (isquot) continue; if (delims.find(string[i]) != -1) { if (start == -1) continue; results ~= string[start..i]; start = -1; continue; } if (start == -1) start = i; } return results; } void startProcess(char[] command) { try { pin = new PipeStream(); pout = new PipeStream(); perr = new PipeStream(); if (fcntl(pin.writeHandle, F_SETFD, 1) == -1) throw new ProcessException("fcntl(pin.writeHandle)"); if (fcntl(pout.readHandle, F_SETFD, 1) == -1) throw new ProcessException("fcntl(pout.readHandle)"); if (fcntl(perr.readHandle, F_SETFD, 1) == -1) throw new ProcessException("fcntl(perr.readHandle)"); if (fcntl(fileno(stdin), F_SETFD, 1) == -1) throw new ProcessException("fcntl(stdin)"); if (fcntl(fileno(stdout), F_SETFD, 1) == -1) throw new ProcessException("fcntl(stdout)"); if (fcntl(fileno(stderr), F_SETFD, 1) == -1) throw new ProcessException("fcntl(stderr)"); pid = fork(); if (pid == 0) { /* child */ //not sure if we can even throw here? if (dup2(pout.writeHandle,STDOUT_FILENO) == -1) {} //throw new ProcessException("dup2(xwrite[1])"); if (dup2(perr.writeHandle,STDERR_FILENO) == -1) {} //throw new ProcessException("dup2(xread[0])"); if (dup2(pin.readHandle,STDIN_FILENO) == -1) {} //throw new ProcessException("dup2(xread[0])"); pout.closeWrite(); perr.closeWrite(); pin.closeRead(); /* set child uid/gid here */ //if (setuid(uid) == -1) throw new ProcessException("setuid"); //if (setgid(gid) == -1) throw new ProcessException("setgid"); execve(args[0],makeBlock(splitArgs(command)),makeBlock(enviroment)); //this does not return on success //can we throw? how to notify parent of failure? exit(1); } /* parent */ running = true; } finally { if (running) { pout.closeWrite(); perr.closeWrite(); pin.closeRead(); } else { pin = null; pout = null; perr = null; } } } void stopProcess(uint dummy) { int r; if (pid == 0) return; if (kill(pid, SIGTERM) == -1) throw new ProcessException("kill"); for(uint i = 0; i < 100; i++) { r = waitpid(pid,null,WNOHANG|WUNTRACED); if (r == -1) throw new ProcessException("waitpid"); if (r == pid) break; usleep(50000); } running = false; close(output); close(input); pid = 0; } } } /+ void main() { auto Process p = new Process("cmd /c dir"); printf("%.*s",p.readLine()); } +/ ------------CYOrmsTJMqxCFBdwjNLfOY--
Apr 05 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Regan Heath wrote:
 On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,  
 command line tool and get its output.  Either directly from the tool's  
 stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because it  
 opens a new command line window when it starts the tool.  That you can  
 get it to close the window again really fast by using 'start' is not  
 good enough
  It doesn't have to be a portable way.  If someone can tell me how to  
 call _popen or something in msvcrt.dll, I would be happy.

I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan

Here they/it is :) Regan

Thanks! But it doesn't work for me. Seems that it blocks the new process from acessing the network. Any ideas what to do?
Apr 05 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,  
 command line tool and get its output.  Either directly from the tool's  
 stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because it  
 opens a new command line window when it starts the tool.  That you can  
 get it to close the window again really fast by using 'start' is not  
 good enough
  It doesn't have to be a portable way.  If someone can tell me how to  
 call _popen or something in msvcrt.dll, I would be happy.

I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan

Here they/it is :) Regan

Thanks! But it doesn't work for me. Seems that it blocks the new process from acessing the network. Any ideas what to do?

Here's a minimal test to show the problem: import lib.process; import std.stdio; void main() { Process proc; proc = new Process("ping google.com"); writef(proc.readLine()); } Compile: "dmd test.d lib/process.d lib/pipestream.d" Output: "Ping request could not find host google.com. Please check the name and try again." ;)
Apr 05 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 05 Apr 2006 19:22:25 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
 On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,   
 command line tool and get its output.  Either directly from the  
 tool's  stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because  
 it  opens a new command line window when it starts the tool.  That  
 you can  get it to close the window again really fast by using  
 'start' is not  good enough
  It doesn't have to be a portable way.  If someone can tell me how  
 to  call _popen or something in msvcrt.dll, I would be happy.

I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan

Regan

But it doesn't work for me. Seems that it blocks the new process from acessing the network. Any ideas what to do?

Here's a minimal test to show the problem: import lib.process; import std.stdio; void main() { Process proc; proc = new Process("ping google.com"); writef(proc.readLine()); } Compile: "dmd test.d lib/process.d lib/pipestream.d"

 Output: "Ping request could not find host google.com. Please check the  
 name and try again."

(Cross posted to digitalmars.D so more people see it) (You have to move the readLine function from private to public in pipestream, that was obviously a mistake on my part) I get the same result. Odd. I thought at first it might be because ping uses enviroment variables to find/use the DNS service, but adding the current enviroment variables to the new process makes no difference: import lib.process; import std.stdio; import std.string; import std.c.string; extern(C) extern char **_environ; void main() { Process proc; proc = new Process(); for(int i = 0; _environ[i]; i++) { proc.addEnv(toString(_environ[i]).dup); } proc.execute("ping www.google.com"); writef(proc.readLine()); } I also tried using CreateProcessAsUser with the handle returned by: OpenProcessToken(GetCurrentProcess(), ..etc.. it made no difference (not that I expected it to, but you never know). Does anyone have any idea why ping cannot access the DNS service? Regan
Apr 05 2006
parent reply Stuart Delaney <Stuart_member pathlink.com> writes:
There's a bug in the makeBlock functions in process.d. The first parameter to
calloc should be 1 not 0. With that change (and the private readLine one) it
works fine for me.  Don't have an answer to the OP's DNS problem though.

hope this helps. cheers,
Stu

In article <ops7j4wey023k2f5 nrage.netwin.co.nz>, Regan Heath says...
On Wed, 05 Apr 2006 19:22:25 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
 On Wed, 05 Apr 2006 00:36:45 -0700, kris <foo bar.com> wrote:
 Tydr Schnubbis wrote:
 I'm making a gui app with DWT, and I need a way to run an external,   
 command line tool and get its output.  Either directly from the  
 tool's  stdout, or by having it write to a file first.
  I'm working on windows.  The system() function is unusable because  
 it  opens a new command line window when it starts the tool.  That  
 you can  get it to close the window again really fast by using  
 'start' is not  good enough
  It doesn't have to be a portable way.  If someone can tell me how  
 to  call _popen or something in msvcrt.dll, I would be happy.

I understand Regan posted a module to do exactly what you want. Maybe he'll see this, or you may be able to dig it up again from the archives (or via google upon the digitalmars site). Reckon the keywords would be something like ~ win32 pipe process regan

Regan

But it doesn't work for me. Seems that it blocks the new process from acessing the network. Any ideas what to do?

Here's a minimal test to show the problem: import lib.process; import std.stdio; void main() { Process proc; proc = new Process("ping google.com"); writef(proc.readLine()); } Compile: "dmd test.d lib/process.d lib/pipestream.d"

 Output: "Ping request could not find host google.com. Please check the  
 name and try again."

(Cross posted to digitalmars.D so more people see it) (You have to move the readLine function from private to public in pipestream, that was obviously a mistake on my part) I get the same result. Odd. I thought at first it might be because ping uses enviroment variables to find/use the DNS service, but adding the current enviroment variables to the new process makes no difference: import lib.process; import std.stdio; import std.string; import std.c.string; extern(C) extern char **_environ; void main() { Process proc; proc = new Process(); for(int i = 0; _environ[i]; i++) { proc.addEnv(toString(_environ[i]).dup); } proc.execute("ping www.google.com"); writef(proc.readLine()); } I also tried using CreateProcessAsUser with the handle returned by: OpenProcessToken(GetCurrentProcess(), ..etc.. it made no difference (not that I expected it to, but you never know). Does anyone have any idea why ping cannot access the DNS service? Regan

Apr 06 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney  
<Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first  
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine  
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem though.

You're dead right. With those changes it works for me too. Regan
Apr 06 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney  
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first  
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine  
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem though.

You're dead right. With those changes it works for me too.

I've fixed the calloc calls too, but it doesn't help. If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.
Apr 06 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney  
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first  
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine  
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem though.

You're dead right. With those changes it works for me too.

I've fixed the calloc calls too, but it doesn't help. If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.

win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network? Where would I start looking if I want to fix this? The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'. But it seems to be the same function. Here's what the call looks like: CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)
Apr 18 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
 Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney   
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first   
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine   
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem  
 though.


ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.

win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network? Where would I start looking if I want to fix this? The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'. But it seems to be the same function. Here's what the call looks like: CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)

CreateProcessA is the ascii version. CreateProcessW is the unicode (UTF-16) version. std.string.toStringz converts the UTF-8 char[] into ascii std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if you want to call CreateProcessW instead. The common cause of the "4invalid UTF-8 sequence" error is trying to output non-ascii characters to the windows console. Can you post your current code here. Regan
Apr 18 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Regan Heath wrote:
 On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake address.dude>  
 wrote:
 Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney   
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first   
 parameter to
 calloc should be 1 not 0. With that change (and the private readLine   
 one) it
 works fine for me.  Don't have an answer to the OP's DNS problem  
 though.


ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.

win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network? Where would I start looking if I want to fix this? The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'. But it seems to be the same function. Here's what the call looks like: CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)

CreateProcessA is the ascii version. CreateProcessW is the unicode (UTF-16) version. std.string.toStringz converts the UTF-8 char[] into ascii std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if you want to call CreateProcessW instead. The common cause of the "4invalid UTF-8 sequence" error is trying to output non-ascii characters to the windows console. Can you post your current code here.

Sure. As you can see it's full of weird characters... import lib.process; import std.stdio; void main() { Process proc; // ping my router proc = new Process("ping 192.168.0.1"); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); } I have only made readLine public, and fixed the four calloc calls, no other changes have been made to your files. Compile with: dmd test.d lib/process.d lib/pipestream.d This prints: Pinging Error: 4invalid UTF-8 sequence If I ping google.com instead, I get this: Ping request could not find host google.com. Please check the name and try again . Error: ReadFile: The pipe has been ended. --------- Does both of these work for you? I have no idea what would cause any of these problems. I have winxp SP2 US. CreateProcessA is in kernel32.dll, of which I have version 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158). dmd version 0.154, got the same results with 0.148. :/
Apr 19 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 19 Apr 2006 16:53:18 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
 Regan Heath wrote:
 On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake address.dude>   
 wrote:
 Tydr Schnubbis wrote:
 Regan Heath wrote:
 On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney    
 <Stuart_member pathlink.com> wrote:
 There's a bug in the makeBlock functions in process.d. The first    
 parameter to
 calloc should be 1 not 0. With that change (and the private  
 readLine   one) it
 works fine for me.  Don't have an answer to the OP's DNS problem   
 though.


ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence". Maybe my windows installation is screwed, it's getting really old... Btw, I use dmd 0.148, haven't tried this with any other version yet.

the win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network? Where would I start looking if I want to fix this? The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'. But it seems to be the same function. Here's what the call looks like: CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)

CreateProcessW is the unicode (UTF-16) version. std.string.toStringz converts the UTF-8 char[] into ascii std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if you want to call CreateProcessW instead. The common cause of the "4invalid UTF-8 sequence" error is trying to output non-ascii characters to the windows console. Can you post your current code here.

Sure. As you can see it's full of weird characters... import lib.process; import std.stdio; void main() { Process proc; // ping my router proc = new Process("ping 192.168.0.1"); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); } I have only made readLine public, and fixed the four calloc calls, no other changes have been made to your files. Compile with: dmd test.d lib/process.d lib/pipestream.d This prints: Pinging Error: 4invalid UTF-8 sequence If I ping google.com instead, I get this: Ping request could not find host google.com. Please check the name and try again . Error: ReadFile: The pipe has been ended. --------- Does both of these work for you? I have no idea what would cause any of these problems. I have winxp SP2 US. CreateProcessA is in kernel32.dll, of which I have version 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158). dmd version 0.154, got the same results with 0.148.

It's quite odd, try this: import lib.process; import std.stdio; import std.string; import std.c.string; extern(C) extern char **_environ; void main() { Process proc; proc = new Process(); for(int i = 0; _environ[i]; i++) { proc.addEnv(toString(_environ[i]).dup); } //proc.execute("ping www.google.com"); proc.execute("ping 192.168.0.1"); while(true) writefln("%s",proc.readLine()); } without the addEnv calls above I get the behaviour you're describing. With them it works. Without them, and using printf I can see that ping responds with: "Pinging with 32 bytes of data:" note the weird characters there. At first I thought maybe the command line I was passing to CreateProcessA was temporary and being collected by the GC, so I changed process.d to use: cmd = strdup(std.string.toStringz(command)); where cmd is a member of Process - so will persist as long as it does. That made no difference. I have no idea why it's doing that, perhaps it reads it's args in a strange way?? I might write a debug program and run that passing different args etc to see if I can replicate the odd behaviour and figure out where it comes from. Regan
Apr 19 2006
parent reply Tydr Schnubbis <fake address.dude> writes:
Regan Heath wrote:
  The common cause of the "4invalid UTF-8 sequence" error is trying to   
 output non-ascii characters to the windows console. Can you post your   
 current code here.

Sure. As you can see it's full of weird characters... import lib.process; import std.stdio; void main() { Process proc; // ping my router proc = new Process("ping 192.168.0.1"); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); writefln(proc.readLine()); } I have only made readLine public, and fixed the four calloc calls, no other changes have been made to your files. Compile with: dmd test.d lib/process.d lib/pipestream.d This prints: Pinging Error: 4invalid UTF-8 sequence If I ping google.com instead, I get this: Ping request could not find host google.com. Please check the name and try again . Error: ReadFile: The pipe has been ended. --------- Does both of these work for you? I have no idea what would cause any of these problems. I have winxp SP2 US. CreateProcessA is in kernel32.dll, of which I have version 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158). dmd version 0.154, got the same results with 0.148.

It's quite odd, try this: import lib.process; import std.stdio; import std.string; import std.c.string; extern(C) extern char **_environ; void main() { Process proc; proc = new Process(); for(int i = 0; _environ[i]; i++) { proc.addEnv(toString(_environ[i]).dup); } //proc.execute("ping www.google.com"); proc.execute("ping 192.168.0.1"); while(true) writefln("%s",proc.readLine()); } without the addEnv calls above I get the behaviour you're describing. With them it works.

 Without them, and using printf I can see that ping responds with:
 
 "Pinging  with 32 bytes of data:"
 
 note the weird characters there. At first I thought maybe the command line  
 I was passing to CreateProcessA was temporary and being collected by the  
 GC, so I changed process.d to use:
 
 cmd = strdup(std.string.toStringz(command));
 
 where cmd is a member of Process - so will persist as long as it does.  
 That made no difference. I have no idea why it's doing that, perhaps it  
 reads it's args in a strange way?? I might write a debug program and run  
 that passing different args etc to see if I can replicate the odd  
 behaviour and figure out where it comes from.

Not sure if this helps: http://www.digitalmars.com/techtips/windows_utf.html
Apr 19 2006
parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 20 Apr 2006 00:48:55 +0200, Tydr Schnubbis <fake address.dude>  
wrote:
  without the addEnv calls above I get the behaviour you're describing.  
 With  them it works.


NP.
 Without them, and using printf I can see that ping responds with:
  "Pinging  with 32 bytes of data:"
  note the weird characters there. At first I thought maybe the command  
 line  I was passing to CreateProcessA was temporary and being collected  
 by the  GC, so I changed process.d to use:
  cmd = strdup(std.string.toStringz(command));
  where cmd is a member of Process - so will persist as long as it  
 does.  That made no difference. I have no idea why it's doing that,  
 perhaps it  reads it's args in a strange way?? I might write a debug  
 program and run  that passing different args etc to see if I can  
 replicate the odd  behaviour and figure out where it comes from.

Not sure if this helps: http://www.digitalmars.com/techtips/windows_utf.html

Nope :( It's my understanding that if you're using ASCII, as we are, you can call the A functions without any conversion, you simply need to ensure there is a null terminater on the string (which is what toStringz does). In any case I tried both toMBSz with CreateProcessA and toUTF16 with CreateProcessW, it made no difference. Those results plus the same ones I got dup'ing the command value suggest to me that the command we're passing isn't the problem. Further, the complete output from ping later shows the parameter correctly, see: "Pinging with 32 bytes of data: Reply from 192.168.1.1: bytes=32 time=5ms TTL=255" (which also means ping can operate without enviroment vars provided it does not need to do a DNS lookup on the name you give it, in this case an ip) I think there may be a bug in ping .. as much as I hate to suggest it (because in most cases you later find out you're wrong). I suspect it somehow uses an enviroment variable without error checking when printing that first line, resulting in garbage being printed. Regan
Apr 19 2006