www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - piping processes

reply "NMS" <nathanmswan gmail.com> writes:
Is there a cross-platform way to create a new process and get its i/o 
streams? Like java.lang.Process? 
Nov 30 2011
next sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 30 Nov 2011 23:35:58 -0000, NMS <nathanmswan gmail.com> wrote:

 Is there a cross-platform way to create a new process and get its i/o  
 streams? Like java.lang.Process?

No. It different on windows and unix platforms, tho most/many of the unix platforms are similar. std.process is rather limited in it's current incarnation but I think Steve is working on a bit update.. I have misplaced the pipestream and processstream code I wrote ages ago which did what you want, otherwise I'd just send it to you. So, best I can do is describe the process. On windows you call Win32 functions, requiring you link with the appropriate windows libs - to find out which ones search for the function names I am about to describe in MSDN online. To start a child process with input handles.. - You call CreatePipe for each input (stdin, stdout, stderr) this gives you 2 handles for each pipe, one handle is for the child process, the other for the parent (think of the cans on string you used to use as a kid) - Pro tip; you now want to call DuplicateHandle on the parent handle for each pipe to remove 'inheritance' - this stops the child process inheriting the handle (which would result in 2 open handles to the pipe, and closure of the parent handle would not be detected by the child). After duplication you close the original parent side inheritable handles. - Call CreateProcess to start the process. In flags specify STARTF_USESTDHANDLES and assign the child end of each of the pipes to each input. - Close the child pipe handles (in the parent) that were passed to CreateProcess. To read from the child - Call PeekNamedPipe or ReadFile on the parent end of the pipe that was passed to the child as it's stdout. To write to the child - Call WriteFile on the parent end of the pipe that was passed to the child as it's stdin. To detect the child termination - CreateProcess returns a thread handle and a process handle, you can wait on either with WaitForSingleObject (I typically use the process handle and close the thread handle immediately after starting the child). I haven't done this sort of thing on unix for ages and I have no good code to hand so someone else will have to help you out with this.. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Dec 01 2011
parent Dejan Lekic <dejan.lekic gmail.com> writes:
We talked about that too on IRC - the conclusion was - we should have 
std.pipe module with (at least) AnonymousPipe, and NamedPipe classes.
Dec 01 2011
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
NMS wrote:

 Is there a cross-platform way to create a new process and get its i/o
 streams? Like java.lang.Process?

I doubt. However, you can use platform-specific popen() (POSIX) and _popen() (Windows) for that. I recently talked about this on IRC. At the moment Phobos uses system() , redirects the output of a command to a temp file, and reads the file (that is how the shell() is implemented on Windows). My question was - why not using _popen() ???
Dec 01 2011
next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 01 Dec 2011 11:29:52 -0000, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:

 I can't find my thread in the archives yet, but I made a similar
 inquiry a few days ago.

 You can see a few examples for Windows here:
 https://github.com/AndrejMitrovic/DWinProgramming/tree/master/Samples/Extra/RedirectChildHandle
 https://github.com/AndrejMitrovic/DWinProgramming/tree/master/Samples/Extra/RedirectChildHandleThreaded

Oddly I haven't used the SetHandleInformation function for this.. I wonder if it arrived /after/ I learnt all this :p I know these are basic examples, but I think in RedirectChildHandle.d, CreateChildProcess, you should (ideally) be closing childStdoutWrite and childStdinRead /after/ CreateProcess. If you don't you get 2 copies of them, one in the child and one in the parent. If the child closes it's copy the parent will not notice the pipe close (when reading from the other end of the pipe - for example). If you close it, it will mean that you will actually drop out of a blocking read in the parent immediately (when the child terminates), rather than timing out and then discovering the child has terminated (using the thread or process handle). Regan -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Dec 01 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/1/11, Regan Heath <regan netmail.co.nz> wrote:
 I know these are basic examples, but I think in RedirectChildHandle.d,
 CreateChildProcess, you should (ideally) be closing childStdoutWrite and
 childStdinRead /after/ CreateProcess.  If you don't you get 2 copies of
 them, one in the child and one in the parent.  If the child closes it's
 copy the parent will not notice the pipe close (when reading from the
 other end of the pipe - for example).  If you close it, it will mean that
 you will actually drop out of a blocking read in the parent immediately
 (when the child terminates), rather than timing out and then discovering
 the child has terminated (using the thread or process handle).

That example comes straight from MSDN's documentation, although I might have made some modifications. Thanks for the info though, I'll see about fixing it.
Dec 01 2011
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
 On 12/1/11, Regan Heath <regan netmail.co.nz> wrote:
 you should (ideally) be closing childStdoutWrite and
 childStdinRead /after/ CreateProcess.  If you don't you get 2 copies of
 them, one in the child and one in the parent.


Ok so I should move CloseHandle(childStdoutWrite) immediately after a successful call to CreateProcess? And add CloseHandle for childStdinRead. I've noticed now childStdinRead was never even closed.
Dec 01 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I can't find my thread in the archives yet, but I made a similar
inquiry a few days ago.

You can see a few examples for Windows here:
https://github.com/AndrejMitrovic/DWinProgramming/tree/master/Samples/Extra/RedirectChildHandle
https://github.com/AndrejMitrovic/DWinProgramming/tree/master/Samples/Extra/RedirectChildHandleThreaded
Dec 01 2011
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 01 Dec 2011 11:28:50 -0000, Dejan Lekic <dejan.lekic gmail.com>  
wrote:

 We talked about that too on IRC - the conclusion was - we should have
 std.pipe module with (at least) AnonymousPipe, and NamedPipe classes.

Definitely. I will have a hunt for my misplaced code but it had some sort of pipe class which was used by the subprocess class or similar. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Dec 01 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 01 Dec 2011 06:10:00 -0500, Regan Heath <regan netmail.co.nz>  
wrote:

 On Wed, 30 Nov 2011 23:35:58 -0000, NMS <nathanmswan gmail.com> wrote:

 Is there a cross-platform way to create a new process and get its i/o  
 streams? Like java.lang.Process?

No. It different on windows and unix platforms, tho most/many of the unix platforms are similar. std.process is rather limited in it's current incarnation but I think Steve is working on a bit update..

It's ready for review, but I'm waiting on myself to get a pull request in for dmc... Going to kick myself to do it now.
   - Pro tip; you now want to call DuplicateHandle on the parent handle  
 for each pipe to remove 'inheritance' - this stops the child process  
 inheriting the handle (which would result in 2 open handles to the pipe,  
 and closure of the parent handle would not be detected by the child).   
 After duplication you close the original parent side inheritable handles.

There's also SetHandleInformation (which I forgot to do in my implementation, thanks for bringing this up) http://msdn.microsoft.com/en-us/library/windows/desktop/ms724935(v=vs.85).aspx -Steve
Dec 01 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 01 Dec 2011 08:24:25 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Thu, 01 Dec 2011 06:10:00 -0500, Regan Heath <regan netmail.co.nz>  
 wrote:
 std.process is rather limited in it's current incarnation but I think  
 Steve is working on a bit update..

It's ready for review, but I'm waiting on myself to get a pull request in for dmc... Going to kick myself to do it now.

Just done, we'll see if Walter merges anytime soon. -Steve
Dec 01 2011