www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Pipe one shell command into another

reply Andrew <aabrown24 hotmail.com> writes:
Hi,

I'd like to run a shell command which involves piping one thing 
into another and then processes the output line by line, i.e. 
something like "ls -l | sort -k5,5n"

What I've come up so far with is:

import std.process;
import std.stdio;

void main(){
   auto pipesLs = pipeProcess(["ls", "-l"], Redirect.stdout);
   auto pipesSort = pipeProcess(["sort", "-k5,5n"], Redirect.all);

   scope (exit) wait(pipesSort.pid);

   foreach(line; pipesLs.stdout.byLine)
     pipesSort.stdin.writeln(line);

   pipesSort.stdin.close;

   foreach(line; pipesSort.stdout.byLine)
     writeln(line);
}

This seems to work on this simple example, but is there a better 
way to do it, and if not, is this reliable? Also, could someone 
explain to me the necessity of the wait command?

Thanks very much

Andrew
Jan 30
parent reply Griffon26 <griffon26 kfk4ever.com> writes:
On Saturday, 30 January 2016 at 15:12:26 UTC, Andrew wrote:
   foreach(line; pipesLs.stdout.byLine)
     pipesSort.stdin.writeln(line);
Because you write sort's input first and read its output later, it might end up blocking if ls generates too much data. The output pipe of sort will fill up, causing sort to block and not read data from its input pipe anymore, resulting in your program blocking when the input pipe has filled up. I have a piece of code using poll to write to two input pipes and read from one output pipe, but it's a bit more complex than what you need. However, it also uses pipeShell to specify multiple commands at once. Since your program only needs to read the output (and not generate input), you may find pipeShell useful. https://github.com/Griffon26/tdiff3/blob/87709dd51c279e9896f37ba1cef477a525e44562/source/gnudiff.d Disclaimer: I'm not an experienced D programmer. My D code may not be pretty =)
Jan 30
parent Andrew <aabrown24 hotmail.com> writes:
On Saturday, 30 January 2016 at 15:57:49 UTC, Griffon26 wrote:
 On Saturday, 30 January 2016 at 15:12:26 UTC, Andrew wrote:
   foreach(line; pipesLs.stdout.byLine)
     pipesSort.stdin.writeln(line);
Because you write sort's input first and read its output later, it might end up blocking if ls generates too much data. The output pipe of sort will fill up, causing sort to block and not read data from its input pipe anymore, resulting in your program blocking when the input pipe has filled up. I have a piece of code using poll to write to two input pipes and read from one output pipe, but it's a bit more complex than what you need. However, it also uses pipeShell to specify multiple commands at once. Since your program only needs to read the output (and not generate input), you may find pipeShell useful. https://github.com/Griffon26/tdiff3/blob/87709dd51c279e9896f37ba1cef477a525e44562/source/gnudiff.d Disclaimer: I'm not an experienced D programmer. My D code may not be pretty =)
pipeShell did the trick. Thank you very much! Andrew
Jan 30