www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Re: forks/pipes and std.socket

reply Kagamin <spam here.lot> writes:
Nick Sabalausky Wrote:

 Does anyone who's done this sort of thing in D before, on Win or Lin, know 
 of anything else in particular to be aware of? 

There's no fork on windows. If you want a multithreaded server, it's usually implemented with threads on windows.
Sep 07 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Kagamin" <spam here.lot> wrote in message 
news:i660qi$nud$1 digitalmars.com...
 Nick Sabalausky Wrote:

 Does anyone who's done this sort of thing in D before, on Win or Lin, 
 know
 of anything else in particular to be aware of?

There's no fork on windows. If you want a multithreaded server, it's usually implemented with threads on windows.

That was just an example. CreateProcess/spawn/system are fine for my purposes. I'm just talking about creating a child process and communicating with it via pipes.
Sep 07 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 07 Sep 2010 14:54:46 -0400, Nick Sabalausky <a a.a> wrote:

 "Kagamin" <spam here.lot> wrote in message
 news:i660qi$nud$1 digitalmars.com...
 Nick Sabalausky Wrote:

 Does anyone who's done this sort of thing in D before, on Win or Lin,
 know
 of anything else in particular to be aware of?

There's no fork on windows. If you want a multithreaded server, it's usually implemented with threads on windows.

That was just an example. CreateProcess/spawn/system are fine for my purposes. I'm just talking about creating a child process and communicating with it via pipes.

The upcoming std.process changes will make this easy. -Steve
Sep 07 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:op.vioibdr2eav7ka localhost.localdomain...
 On Tue, 07 Sep 2010 14:54:46 -0400, Nick Sabalausky <a a.a> wrote:

 "Kagamin" <spam here.lot> wrote in message
 news:i660qi$nud$1 digitalmars.com...
 Nick Sabalausky Wrote:

 Does anyone who's done this sort of thing in D before, on Win or Lin,
 know
 of anything else in particular to be aware of?

There's no fork on windows. If you want a multithreaded server, it's usually implemented with threads on windows.

That was just an example. CreateProcess/spawn/system are fine for my purposes. I'm just talking about creating a child process and communicating with it via pipes.

The upcoming std.process changes will make this easy.

Ah cool, looking forward to it. I was just about ready to launch into a bunch of std.process improvements myself ;) In the meantime, it seems that File has an undocumented constructor that takes a file handle instead of a filename, so that should work for me. BTW, do any of the upcoming std.process changes do anything to help work around Windows's exec being broken? By that I mean, exec is supposed to *reuse* the current process when launching the new one, but on Windows (and I *don't* believe this is D-specific) it creates a *new* process and kills the old one, which fucks up anything that waits on the original process to finish (such as the command-line). spawn(OVERLAY) has the same problem.
Sep 07 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:op.vipr20xfeav7ka localhost.localdomain...
 On Tue, 07 Sep 2010 16:51:48 -0400, Nick Sabalausky <a a.a> wrote:

 Ah cool, looking forward to it. I was just about ready to launch into a
 bunch of std.process improvements myself ;) In the meantime, it seems 
 that
 File has an undocumented constructor that takes a file handle instead of 
 a
 filename, so that should work for me.

Beware, it takes a file *descriptor*, which is different from a *HANDLE* on windows. In writing the updated std.process I had to write D code that mimics the internals of DMC's runtime in order to translate between file descriptors and handles, not a pretty sight...

I'm just feeding it the values I get from CreatePipe: http://msdn.microsoft.com/en-us/library/aa365152(VS.85).aspx Seems to be working fine, though...
 The updates are being held back right now by bug 3979.

 BTW, do any of the upcoming std.process changes do anything to help work
 around Windows's exec being broken? By that I mean, exec is supposed to
 *reuse* the current process when launching the new one, but on Windows 
 (and
 I *don't* believe this is D-specific) it creates a *new* process and 
 kills
 the old one, which fucks up anything that waits on the original process 
 to
 finish (such as the command-line). spawn(OVERLAY) has the same problem.

Windows' exec is a hack. std.process will not use it, instead, we'll use CreateProcess directly.

Yea, I've come to the conclusion, too, that exec and spawn(OVERLAY) are worthless on Windows. I was going to create and submit a patch that avoids them and uses system instead (which *does* maintain the current pid) and then exits. But if someone's already been doing this, and using CreateProcess directly, then all the better :)
 FWIW, we are not going to support fork or exec individually in 
 std.process.  We will only support creating a new process via fork+exec or 
 CreateProcess.  This is the common functionality among OSes.

 If you want fork or exec individually, you have to use them on posix 
 systems only, and call them directly.

Sounds good.
Sep 09 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:op.vir82cineav7ka localhost.localdomain...
 On Thu, 09 Sep 2010 16:25:44 -0400, Nick Sabalausky <a a.a> wrote:

 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in message
 news:op.vipr20xfeav7ka localhost.localdomain...
 On Tue, 07 Sep 2010 16:51:48 -0400, Nick Sabalausky <a a.a> wrote:

 Ah cool, looking forward to it. I was just about ready to launch into a
 bunch of std.process improvements myself ;) In the meantime, it seems
 that
 File has an undocumented constructor that takes a file handle instead 
 of
 a
 filename, so that should work for me.

Beware, it takes a file *descriptor*, which is different from a *HANDLE* on windows. In writing the updated std.process I had to write D code that mimics the internals of DMC's runtime in order to translate between file descriptors and handles, not a pretty sight...

I'm just feeding it the values I get from CreatePipe: http://msdn.microsoft.com/en-us/library/aa365152(VS.85).aspx Seems to be working fine, though...

That won't work. I'm surprised it works at all. DMC maintains the list of handles in an array, with the index of the array being the file descriptor, and the value at that element being the HANDLE. If it's working, you're extremely lucky that the unallocated slot at the index that happens to be that HANDLE value has the same value as the HANDLE itself. Or maybe I'm misunderstanding something... Walter?

Here's a simplified example of what I'm doing: --------------------------------
 type mainapp.d

import std.process; import std.stdio; import std.stream; import core.sys.windows.windows; // From here: http://msdn.microsoft.com/en-us/library/aa365152(VS.85).aspx extern(Windows) int CreatePipe( HANDLE* hReadPipe, HANDLE* hWritePipe, SECURITY_ATTRIBUTES* lpPipeAttributes, uint nSize); void main() { // Set up pipe HANDLE readHandle; HANDLE writeHandle; auto secAttr = SECURITY_ATTRIBUTES(SECURITY_ATTRIBUTES.sizeof, null, true); if(!CreatePipe(&readHandle, &writeHandle, &secAttr, 0)) throw new Exception("Couldn't create pipe"); auto pipeReader = new std.stream.File(readHandle, FileMode.In); // Run subapp system("subapp "~to!string(cast(size_t)writeHandle)); // Retrieve value from pipe int valueFromSubApp; pipeReader.read(valueFromSubApp); writeln("Received: ", valueFromSubApp); }
 type subapp.d

import std.stdio; import std.stream; import std.c.windows.windows; void main(string[] args) { auto writeHandle = cast(HANDLE)std.conv.to!size_t(args[1]); auto pipeWriter = new std.stream.File(writeHandle, FileMode.Out); int value = 777; writeln("Sending: ", value); pipeWriter.write(value); }
 dmd subapp.d
 dmd mainapp.d
 mainapp

Received: 777 -------------------------------- Maybe the OS is just allowing me to use the wrong file descriptor?
Sep 09 2010
parent "Nick Sabalausky" <a a.a> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:op.vitle0kgeav7ka localhost.localdomain...
 On Thu, 09 Sep 2010 18:29:55 -0400, Nick Sabalausky <a a.a> wrote:


 Maybe the OS is just allowing me to use the wrong file descriptor?

OK, after downloading the latest dmd, testing your code, and perplexing a while, then reading the source of phobos, I realized that you are using std.stream.File, not std.stdio.File. The former is being deprecated for the latter. Yes, std.stream.File will work. But std.stdio.File is what is going to be supported. std.stream is scheduled for deprecation.

Ahh ok, I thought it seemed strange that there were both. std.stdio.File is going to allow using a file descriptor, right? It looked like all it currently supports in its public interface was a filename, but maybe I'm remembering wrong. Supposing I did need to convert the handle to a file descriptor, is there already a way to do that, or is that in-the-works?
Sep 10 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 07 Sep 2010 16:51:48 -0400, Nick Sabalausky <a a.a> wrote:

 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in message
 news:op.vioibdr2eav7ka localhost.localdomain...
 On Tue, 07 Sep 2010 14:54:46 -0400, Nick Sabalausky <a a.a> wrote:

 "Kagamin" <spam here.lot> wrote in message
 news:i660qi$nud$1 digitalmars.com...
 Nick Sabalausky Wrote:

 Does anyone who's done this sort of thing in D before, on Win or Lin,
 know
 of anything else in particular to be aware of?

There's no fork on windows. If you want a multithreaded server, it's usually implemented with threads on windows.

That was just an example. CreateProcess/spawn/system are fine for my purposes. I'm just talking about creating a child process and communicating with it via pipes.

The upcoming std.process changes will make this easy.

Ah cool, looking forward to it. I was just about ready to launch into a bunch of std.process improvements myself ;) In the meantime, it seems that File has an undocumented constructor that takes a file handle instead of a filename, so that should work for me.

Beware, it takes a file *descriptor*, which is different from a *HANDLE* on windows. In writing the updated std.process I had to write D code that mimics the internals of DMC's runtime in order to translate between file descriptors and handles, not a pretty sight... The updates are being held back right now by bug 3979.
 BTW, do any of the upcoming std.process changes do anything to help work
 around Windows's exec being broken? By that I mean, exec is supposed to
 *reuse* the current process when launching the new one, but on Windows  
 (and
 I *don't* believe this is D-specific) it creates a *new* process and  
 kills
 the old one, which fucks up anything that waits on the original process  
 to
 finish (such as the command-line). spawn(OVERLAY) has the same problem.

Windows' exec is a hack. std.process will not use it, instead, we'll use CreateProcess directly. FWIW, we are not going to support fork or exec individually in std.process. We will only support creating a new process via fork+exec or CreateProcess. This is the common functionality among OSes. If you want fork or exec individually, you have to use them on posix systems only, and call them directly. -Steve
Sep 08 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 09 Sep 2010 16:25:44 -0400, Nick Sabalausky <a a.a> wrote:

 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in message
 news:op.vipr20xfeav7ka localhost.localdomain...
 On Tue, 07 Sep 2010 16:51:48 -0400, Nick Sabalausky <a a.a> wrote:

 Ah cool, looking forward to it. I was just about ready to launch into a
 bunch of std.process improvements myself ;) In the meantime, it seems
 that
 File has an undocumented constructor that takes a file handle instead  
 of
 a
 filename, so that should work for me.

Beware, it takes a file *descriptor*, which is different from a *HANDLE* on windows. In writing the updated std.process I had to write D code that mimics the internals of DMC's runtime in order to translate between file descriptors and handles, not a pretty sight...

I'm just feeding it the values I get from CreatePipe: http://msdn.microsoft.com/en-us/library/aa365152(VS.85).aspx Seems to be working fine, though...

That won't work. I'm surprised it works at all. DMC maintains the list of handles in an array, with the index of the array being the file descriptor, and the value at that element being the HANDLE. If it's working, you're extremely lucky that the unallocated slot at the index that happens to be that HANDLE value has the same value as the HANDLE itself. Or maybe I'm misunderstanding something... Walter? -Steve
Sep 09 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 09 Sep 2010 18:29:55 -0400, Nick Sabalausky <a a.a> wrote:


 Maybe the OS is just allowing me to use the wrong file descriptor?

OK, after downloading the latest dmd, testing your code, and perplexing a while, then reading the source of phobos, I realized that you are using std.stream.File, not std.stdio.File. The former is being deprecated for the latter. Yes, std.stream.File will work. But std.stdio.File is what is going to be supported. std.stream is scheduled for deprecation. -Steve
Sep 10 2010