digitalmars.D.learn - D: How do I pipe (|) through three programs using std.process?
- BoQsc (28/28) Nov 11 2023 https://dlang.org/library/std/process.html
- kdevel (25/30) Nov 11 2023 ```d
- BoQsc (58/58) Nov 12 2023 On Windows:
- BoQsc (24/24) Nov 12 2023 Using `spawnShell` it all seem to work.
- Adam D Ruppe (10/12) Nov 12 2023 spawnProcess always encodes its arguments in a very specific way
- BoQsc (25/25) Nov 12 2023 To make this thread more complete, here is the final version.
- Jesse Phillips (5/7) Nov 14 2023 https://dev.to/jessekphillips/piping-process-output-1cai
https://dlang.org/library/std/process.html How do I pipe (|) through three programs using std.process? I'm getting confused again about what functions should I use to efficiently mimick piping like it is done in some shells (Linux bash, Windows cmd): Here I will provide something to test against on Windows Operating System. **Shell example on Windows:** ``` echo This is a sample text | find "sample" | find "text" ``` **Output if piping works:** ``` This is a sample text ```  **Output if it does not:** ``` ```  **Explanation:** `echo` is used to print output to stdout. The output is redirected as **input** to a `find` command. The `find "textToFind"` command finds matching text in the stdin input. Windows `find` command: * If text matches it **returns the whole given input**. * If the text does not match, **it returns nothing**.
Nov 11 2023
On Saturday, 11 November 2023 at 17:29:14 UTC, BoQsc wrote:https://dlang.org/library/std/process.html How do I pipe (|) through three programs using std.process? ``` echo This is a sample text | find "sample" | find "text" ``````d import std.stdio; import std.process; version (Windows) { enum Find = "find"; } version (Posix) { enum Find = "grep"; } int main (string [] args) { auto p1 = pipe; auto p2 = pipe; auto pid1 = spawnProcess (args [1..$], stdin, p1.writeEnd); auto pid2 = spawnProcess ([Find, "sample"], p1.readEnd, p2.writeEnd); auto pid3 = spawnProcess ([Find, "text"], p2.readEnd, stdout); wait (pid1); wait (pid2); wait (pid3); return 0; } ``` ``` $ ./pip echo This is a sample text This is a sample text $ ./pip echo This is an ample text ```
Nov 11 2023
On Windows:
While trying to use `spawnshell` I discovered that I can not use
any alphabetical letters inside the `spawnProcess([Find,
"Hello"])` it all works when they are numerical `[Find, "6515"]`.
As of recent testing `[Find, "df123"]` also is acceptable,
but not when letter is on the right `[Find, "df123d"]`
Unable to figure why this is the case and how to resolve it.
**Working example without problems:**
```
import std.stdio;
import std.process;
version (Windows) { enum Find = "find"; }
version (Posix) { enum Find = "grep"; }
int main (string [] args)
{
auto p1 = pipe;
auto p2 = pipe;
auto pid1 = spawnShell("echo 123", stdin, p1.writeEnd);
auto pid2 = spawnProcess([Find, "123"], p1.readEnd,
p2.writeEnd);
auto pid3 = spawnProcess([Find, "123"], p2.readEnd, stdout);
wait (pid1);
wait (pid2);
wait (pid3);
return 0;
}
```
**Output:**
```
123
```
**Immediate issue if alphabetical letters are used:**
```
import std.stdio;
import std.process;
version (Windows) { enum Find = "find"; }
version (Posix) { enum Find = "grep"; }
int main (string [] args)
{
auto p1 = pipe;
auto p2 = pipe;
auto pid1 = spawnShell("echo HelloWorld", stdin, p1.writeEnd);
auto pid2 = spawnProcess([Find, "HelloWorld"], p1.readEnd,
p2.writeEnd);
auto pid3 = spawnProcess([Find, "HelloWorld"], p2.readEnd,
stdout);
wait (pid1);
wait (pid2);
wait (pid3);
return 0;
}
```
**Output:**
```
FIND: Parameter format not correct
FIND: Parameter format not correct
The process tried to write to a nonexistent pipe.
```
Nov 12 2023
Using `spawnShell` it all seem to work.
However the question of why `spawnProcess(["find", "string to
find"]` is not working and produces error is still unresolved.
Works with `spawnShell`:
```
import std.stdio;
import std.process;
version (Windows) { enum Find = "find"; }
version (Posix) { enum Find = "grep"; }
int main (string [] args)
{
auto p1 = pipe;
auto p2 = pipe;
auto pid1 = spawnShell("echo HelloWorld", stdin, p1.writeEnd);
auto pid2 = spawnShell("find \"HelloWorld\"", p1.readEnd,
p2.writeEnd);
auto pid3 = spawnShell("find \"HelloWorld\"", p2.readEnd,
stdout);
wait (pid1);
wait (pid2);
wait (pid3);
return 0;
}
```
Nov 12 2023
Adam D Ruppe <destructionator gmail.com> On Sunday, 12 November 2023 at 13:39:25 UTC, BoQsc wrote:However the question of why `spawnProcess(["find", "string to find"]` is not working and produces error is still unresolved.spawnProcess always encodes its arguments in a very specific way and the receiving programs are not always compatible with that thing. A Windows process does not take an array of args, but rather a single string command line. spawnProcess tries to turn the unix-style array into a single string that can then be turned back into arguments by the receiving program. But it makes a lot of assumptions in how that happens that just don't always match reality.
Nov 12 2023
To make this thread more complete, here is the final version.
```
import std.stdio;
import std.process;
version (Windows) { enum Find = "find"; }
version (Posix) { enum Find = "grep"; }
int main (string [] args)
{
auto p1 = pipe;
auto p2 = pipe;
auto pid1 = spawnShell("echo HelloWorld", stdin, p1.writeEnd);
auto pid2 = spawnShell(Find ~ " \"HelloWorld\"", p1.readEnd,
p2.writeEnd);
auto pid3 = spawnShell(Find ~ " \"HelloWorld\"", p2.readEnd,
stdout);
wait (pid1);
wait (pid2);
wait (pid3);
return 0;
}
```
It is equal to:
* Windows: `echo HelloWorld | find "HelloWorld | find
"HelloWorld"`
* Linux: `echo HelloWorld | grep "HelloWorld | grep "HelloWorld"`
Nov 12 2023
On Saturday, 11 November 2023 at 17:29:14 UTC, BoQsc wrote:https://dlang.org/library/std/process.html How do I pipe (|) through three programs using std.process?https://dev.to/jessekphillips/piping-process-output-1cai Your issue with [Find, "Hello"] might be [Find, "\"Hello\""] But I'm not testing the theory...
Nov 14 2023
Latest iteration on this thread.
Limitations:
* pipes through two programs.
* very verbose, hard to use.
```
import std;
import std.process;
version (Windows) { enum Find = "find"; }
version (Posix) { enum Find = "grep"; }
void pipeTo(Pipe p, string nextprogram){
spawnShell(nextprogram, p.readEnd, stdout);
}
auto program(string name){
Pipe p = std.process.pipe;
spawnShell(name, stdin, p.writeEnd);
return p;
}
void main()
{
program("echo HelloWorld").pipeTo(nextprogram: Find ~ `
"HelloWorld"`);
}
```
Nov 18 2023
On Saturday, 18 November 2023 at 18:09:53 UTC, BoQsc wrote:Latest iteration on this thread. Limitations: * pipes through two programs. * very verbose, hard to use.What exactly are you trying to achieve?``` import std; import std.process; version (Windows) { enum Find = "find"; } version (Posix) { enum Find = "grep"; } void pipeTo(Pipe p, string nextprogram){ spawnShell(nextprogram, p.readEnd, stdout); ```If you allow invoking the shell from within your program why don't you use one of its facilities, i.e. the shell's pipe operator `|`, in the first place? `spawnShell` does not execute `nextprogram` but it executes it under the shell.``` } auto program(string name){ Pipe p = std.process.pipe; spawnShell(name, stdin, p.writeEnd); return p; } void main() { program("echo HelloWorld").pipeTo(nextprogram: Find ~ ` "HelloWorld"`);Have I missed the advent of named function arguments in D?``` } ```Your whole program shrinks considerably: ``` import std; import std.process; version (Windows) { enum Find = "find"; } version (Posix) { enum Find = "grep"; } void main() { spawnShell("echo HelloWorld | " ~ Find ~ ` "HelloWorld"`).wait; } ```
Nov 19 2023