www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Suggestion for std.process upgrade

reply Bane <branimir.milosavljevic gmail.com> writes:
I have been playing with std.process (D2) lately, and have 2 suggestions 
and more or less tested code if somebody other than my self have use for it.

One is shell() function (not mentioned in the docs, curious), I see it 
can be made more efficient on Windows. It executes shell command and 
returns standard output as string. Current implementation do it by 
piping stdout to temporary file on disk and reading that file back. 
Using CreateProccess Windows API it can do same job 3 times faster and 
remove need for temporary files and disk writes. I think that is 
beneficial gain for some applications.

Other is fork-exec implementation eg. starting a program using command 
line and detaching it from parent, so it continues to run after parent 
is dead. On Posix it is implemented using fork() and exec() calls, on 
Windows using CreateProcess.

One question. Is std.c.windows.windows lacking for most API definitions 
or am I looking on wrong place?
Dec 12 2011
next sibling parent Trass3r <un known.com> writes:
 One question. Is std.c.windows.windows lacking for most API definitions  
 or am I looking on wrong place?


Yep, this one is more complete:  
http://dsource.org/projects/bindings/wiki/WindowsApi
But for some reason we still keep that old version, sigh.
Dec 12 2011
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 12 Dec 2011 17:08:53 -0500, Bane  
<branimir.milosavljevic gmail.com> wrote:


 I have been playing with std.process (D2) lately, and have 2 suggestions  
 and more or less tested code if somebody other than my self have use for  
 it.

 One is shell() function (not mentioned in the docs, curious), I see it  
 can be made more efficient on Windows. It executes shell command and  
 returns standard output as string. Current implementation do it by  
 piping stdout to temporary file on disk and reading that file back.  
 Using CreateProccess Windows API it can do same job 3 times faster and  
 remove need for temporary files and disk writes. I think that is  
 beneficial gain for some applications.

 Other is fork-exec implementation eg. starting a program using command  
 line and detaching it from parent, so it continues to run after parent  
 is dead. On Posix it is implemented using fork() and exec() calls, on  
 Windows using CreateProcess.


There is a completely revamped version of std.process.  It is being held  
up right now because DMD on windows depends on DMC for it's C runtime, and  
DMC has issues supporting pipes.  I have recently opened a pull request  
for Walter to merge, I'm going to ping him about it right after 2.057 is  
released.

For more info, see the docs Lars posted here:

http://kyllingen.net/code/ltk/doc/process.html

Once the DMC issue is fixed, you should see this improvement getting much  
more attention.  It's very low hanging fruit.

-Steve
Dec 12 2011
next sibling parent Trass3r <un known.com> writes:
 There is a completely revamped version of std.process.  It is being held  
 up right now because DMD on windows depends on DMC for it's C runtime,  
 and DMC has issues supporting pipes.


Walter should just release the few parts of the source code that according  
to Mehrdad are needed to get rid of it:


 Right, but my point is, I could work around (I've gotten within inches  
 of it!)
 it if simply Walter released a TINY part of snn.lib -- just a handful of  
 tiny source
 files regarding the TLS-related stuff, EXE segment markers, and whatnot.
 (I've already raised this issue before, and precisely what we would  
 need, but it seemed to go completely ignored.)
Dec 12 2011
prev sibling next sibling parent Graham St Jack <Graham.StJack internode.on.net> writes:
On 13/12/11 09:13, Steven Schveighoffer wrote:

 On Mon, 12 Dec 2011 17:08:53 -0500, Bane 
 <branimir.milosavljevic gmail.com> wrote:


 I have been playing with std.process (D2) lately, and have 2 
 suggestions and more or less tested code if somebody other than my 
 self have use for it.

 One is shell() function (not mentioned in the docs, curious), I see 
 it can be made more efficient on Windows. It executes shell command 
 and returns standard output as string. Current implementation do it 
 by piping stdout to temporary file on disk and reading that file 
 back. Using CreateProccess Windows API it can do same job 3 times 
 faster and remove need for temporary files and disk writes. I think 
 that is beneficial gain for some applications.

 Other is fork-exec implementation eg. starting a program using 
 command line and detaching it from parent, so it continues to run 
 after parent is dead. On Posix it is implemented using fork() and 
 exec() calls, on Windows using CreateProcess.


 There is a completely revamped version of std.process.  It is being 
 held up right now because DMD on windows depends on DMC for it's C 
 runtime, and DMC has issues supporting pipes.  I have recently opened 
 a pull request for Walter to merge, I'm going to ping him about it 
 right after 2.057 is released.

 For more info, see the docs Lars posted here:

 http://kyllingen.net/code/ltk/doc/process.html

 Once the DMC issue is fixed, you should see this improvement getting 
 much more attention.  It's very low hanging fruit.

 -Steve


I took a look at the link, and it looks very nice.

I am currently trying to port an application over from Linux to Windows 
and the lack of wait(pid) is currently a blocker.

-- 
Graham St Jack
Dec 12 2011
prev sibling parent Bane <branimir.milosavljevic gmail.com> writes:
 There is a completely revamped version of std.process.  It is being held  
 up right now because DMD on windows depends on DMC for it's C runtime, and  
 DMC has issues supporting pipes.  I have recently opened a pull request  
 for Walter to merge, I'm going to ping him about it right after 2.057 is  
 released.
 
 For more info, see the docs Lars posted here:
 
 http://kyllingen.net/code/ltk/doc/process.html
 
 Once the DMC issue is fixed, you should see this improvement getting much  
 more attention.  It's very low hanging fruit.
 
 -Steve



Nice piece of work. I am looking forward for it becoming part of Phobos.
I find multi-process work interesting alternative to multi-threaded, sometimes
more stable and robust solution.
Dec 12 2011