digitalmars.D.learn - vibe.d - asynchronously wait() for process to exit
- Vladimir Panteleev (14/14) Jun 17 2016 std.process.wait() will wait for a child process to exit and
- cy (10/14) Jun 18 2016 Well, vibe.d streams are defined as interfaces, so you'd have to
- Steven Schveighoffer (11/14) Jun 20 2016 What is the OS support for waitid
- Vladimir Panteleev (4/14) Jun 20 2016 std.process has tryWait() if polling were acceptable, but I would
- Steven Schveighoffer (16/28) Jun 20 2016 tryWait works on a single PID. From my reading of the docs, it appears
- Vladimir Panteleev (25/61) Jun 20 2016 Ah, OK. But then so does SIGCHLD, asynchronously.
- Johannes Pfau (8/14) Jun 21 2016 Such a wrapper would be useful for some more things (inotify/fanotify).
- Steven Schveighoffer (13/64) Jun 21 2016 But not where you need it to be handled :) This is the reason for the
std.process.wait() will wait for a child process to exit and return its exit code. How can this be done in Vibe.d, without blocking other fibers and without creating a new thread? In my library I did it like this: https://github.com/CyberShadow/ae/blob/master/sys/process.d (register a SIGCHLD signal handler, which pings the main thread via a socket). Geod24 on IRC suggested signalfd + createFileDescriptorEvent. I think this would work, but isn't it possible to wrap the fd returned by signalfd into a Vibe.d stream and read it directly? I'm just not sure how. I noticed libasync also provides notification for POSIX signals, but I've no idea where to start with using that in a Vibe.d program.
Jun 17 2016
cy <dlang verge.info.tm> On Friday, 17 June 2016 at 13:53:15 UTC, Vladimir Panteleev wrote:Geod24 on IRC suggested signalfd + createFileDescriptorEvent. I think this would work, but isn't it possible to wrap the fd returned by signalfd into a Vibe.d stream and read it directly? I'm just not sure how.Well, vibe.d streams are defined as interfaces, so you'd have to import vibe.core.stream: InputStream, and create a SignalFdInput class that implemented all the required methods. When it requires you to "wait" for data available, you save core.task.Task.getThis() somewhere, and... basically do what you're doing with createFileDescriptorEvent, just resuming the task instead of handling the event in a callback. I should point out that createFileDescriptorEvent is an assert(0) for libasync.
Jun 18 2016
On 6/17/16 9:53 AM, Vladimir Panteleev wrote:std.process.wait() will wait for a child process to exit and return its exit code. How can this be done in Vibe.d, without blocking other fibers and without creating a new thread?What is the OS support for waitid (http://man7.org/linux/man-pages/man2/waitpid.2.html)? Seems to have support for async waiting of multiple processes (at least it can return immediately if no child has exited). One consideration is how responsive you need to be to a process exiting -- is it ok for example to be notified 500ms after the process exits? If so, you can interleave timed waits for socket data with a check to see if any process exits. I've done this in the past for such things. I don't know how well this works for libevent though. -Steve
Jun 20 2016
On Monday, 20 June 2016 at 16:16:32 UTC, Steven Schveighoffer wrote:What is the OS support for waitid (http://man7.org/linux/man-pages/man2/waitpid.2.html)? Seems to have support for async waiting of multiple processes (at least it can return immediately if no child has exited). One consideration is how responsive you need to be to a process exiting -- is it ok for example to be notified 500ms after the process exits? If so, you can interleave timed waits for socket data with a check to see if any process exits. I've done this in the past for such things. I don't know how well this works for libevent though.std.process has tryWait() if polling were acceptable, but I would really like to avoid it. Or have I misunderstood?
Jun 20 2016
On 6/20/16 12:29 PM, Vladimir Panteleev wrote:On Monday, 20 June 2016 at 16:16:32 UTC, Steven Schveighoffer wrote:tryWait works on a single PID. From my reading of the docs, it appears you can call waitid and no matter how many children you have, if one exits, then it will capture that. It would be nice if the Linux wait mechanisms were all standardized similar to Windows. I think the only thing that allows such universal access is file descriptors. Processes are definitely a case where it's not easy to deal with the events. Signal handlers suck as an async mechanism. But my point was that you can poll on every start of event loop, and handle process exits if they are ready, and then every 500ms or so if no i/o becomes ready. In practice, this should be pretty responsive, unless you are only doing process execution and no i/o. And half second delay between process exit and handling of result is pretty small even in that case. -SteveWhat is the OS support for waitid (http://man7.org/linux/man-pages/man2/waitpid.2.html)? Seems to have support for async waiting of multiple processes (at least it can return immediately if no child has exited). One consideration is how responsive you need to be to a process exiting -- is it ok for example to be notified 500ms after the process exits? If so, you can interleave timed waits for socket data with a check to see if any process exits. I've done this in the past for such things. I don't know how well this works for libevent though.std.process has tryWait() if polling were acceptable, but I would really like to avoid it. Or have I misunderstood?
Jun 20 2016
On Monday, 20 June 2016 at 19:39:42 UTC, Steven Schveighoffer wrote:On 6/20/16 12:29 PM, Vladimir Panteleev wrote:Ah, OK. But then so does SIGCHLD, asynchronously.On Monday, 20 June 2016 at 16:16:32 UTC, Steven Schveighoffer wrote:tryWait works on a single PID. From my reading of the docs, it appears you can call waitid and no matter how many children you have, if one exits, then it will capture that.What is the OS support for waitid (http://man7.org/linux/man-pages/man2/waitpid.2.html)? Seems to have support for async waiting of multiple processes (at least it can return immediately if no child has exited). One consideration is how responsive you need to be to a process exiting -- is it ok for example to be notified 500ms after the process exits? If so, you can interleave timed waits for socket data with a check to see if any process exits. I've done this in the past for such things. I don't know how well this works for libevent though.std.process has tryWait() if polling were acceptable, but I would really like to avoid it. Or have I misunderstood?It would be nice if the Linux wait mechanisms were all standardized similar to Windows. I think the only thing that allows such universal access is file descriptors. Processes are definitely a case where it's not easy to deal with the events. Signal handlers suck as an async mechanism.It's really not that hard. It's just that no one bothered to implement this correctly in Vibe. Process or signal handling does not seem to be a Vibe.d driver primitive. Signals interrupt blocking calls such as select/poll. Even if you don't have a signal handler registered, you could in theory call tryWait or similar on every process ID you're waiting on in an event loop idle handler. It's not very efficient, of course, and degrades poorly as the number of processes and events grows. You can also register a signal handler, and just ping a socket pair (unmanaged on one side, managed by the event loop on the other). This is what I do in ae. As I recently learned, there's also signalfd. With that, had Vibe.d had a primitive to wrap a file descriptor into a stream it can manage, it would be as simple as reading from it. But it doesn't seem to have one so I guess you need to use createFileDescriptorEvent and the raw C read() function.But my point was that you can poll on every start of event loop, and handle process exits if they are ready, and then every 500ms or so if no i/o becomes ready. In practice, this should be pretty responsive, unless you are only doing process execution and no i/o. And half second delay between process exit and handling of result is pretty small even in that case.Sure, but to be honest that's nothing but an ugly hack. :) Even if not for the 500ms delay - every bit adds up, and timers are much worse than event handling on e.g. mobile devices than servers, because they incur a CPU wake-up and then end up doing nothing most of the time. Anyway, I'll probably do one of the above, thanks.
Jun 20 2016
Am Tue, 21 Jun 2016 03:01:39 +0000 schrieb Vladimir Panteleev <thecybershadow.lists gmail.com>:As I recently learned, there's also signalfd. With that, had Vibe.d had a primitive to wrap a file descriptor into a stream it can manage, it would be as simple as reading from it. But it doesn't seem to have one so I guess you need to use createFileDescriptorEvent and the raw C read() function.Such a wrapper would be useful for some more things (inotify/fanotify). Anyway, I wrote such a similar wrapper for a serial port module: https://github.com/jpf91/vibe-serial/blob/master/src/vibe/serial.d#L145 Only reading is fully implemented / tested, but maybe this is still useful. This vibe.d issue could cause problems though: https://github.com/rejectedsoftware/vibe.d/issues/695
Jun 21 2016
On 6/20/16 11:01 PM, Vladimir Panteleev wrote:On Monday, 20 June 2016 at 19:39:42 UTC, Steven Schveighoffer wrote:But not where you need it to be handled :) This is the reason for the "self pipe trick" that you use. I'll note that waitid actually isn't needed, you can do waitpid(-1, ...) and it waits for any process. I didn't realize that before.On 6/20/16 12:29 PM, Vladimir Panteleev wrote:Ah, OK. But then so does SIGCHLD, asynchronously.On Monday, 20 June 2016 at 16:16:32 UTC, Steven Schveighoffer wrote:tryWait works on a single PID. From my reading of the docs, it appears you can call waitid and no matter how many children you have, if one exits, then it will capture that.What is the OS support for waitid (http://man7.org/linux/man-pages/man2/waitpid.2.html)? Seems to have support for async waiting of multiple processes (at least it can return immediately if no child has exited). One consideration is how responsive you need to be to a process exiting -- is it ok for example to be notified 500ms after the process exits? If so, you can interleave timed waits for socket data with a check to see if any process exits. I've done this in the past for such things. I don't know how well this works for libevent though.std.process has tryWait() if polling were acceptable, but I would really like to avoid it. Or have I misunderstood?Only if you handle the signal in that thread.It would be nice if the Linux wait mechanisms were all standardized similar to Windows. I think the only thing that allows such universal access is file descriptors. Processes are definitely a case where it's not easy to deal with the events. Signal handlers suck as an async mechanism.It's really not that hard. It's just that no one bothered to implement this correctly in Vibe. Process or signal handling does not seem to be a Vibe.d driver primitive. Signals interrupt blocking calls such as select/poll. Even if you don't have a signal handler registered, you could in theory call tryWait or similar on every process ID you're waiting on in an event loop idle handler. It's not very efficient, of course, and degrades poorly as the number of processes and events grows.You can also register a signal handler, and just ping a socket pair (unmanaged on one side, managed by the event loop on the other). This is what I do in ae.Yeah, probably the right thing to do is a signal handler that reaps all terminated processes, putting the data into the pipe/socket.As I recently learned, there's also signalfd. With that, had Vibe.d had a primitive to wrap a file descriptor into a stream it can manage, it would be as simple as reading from it. But it doesn't seem to have one so I guess you need to use createFileDescriptorEvent and the raw C read() function.Hm... I hadn't heard of this. Some seem to think that creates its own problems, but I don't know if SIGCHLD is one of them since that's blocked by default: https://ldpreload.com/blog/signalfd-is-uselessWhat are you doing spawning child processes on a mobile device? :) -SteveBut my point was that you can poll on every start of event loop, and handle process exits if they are ready, and then every 500ms or so if no i/o becomes ready. In practice, this should be pretty responsive, unless you are only doing process execution and no i/o. And half second delay between process exit and handling of result is pretty small even in that case.Sure, but to be honest that's nothing but an ugly hack. :) Even if not for the 500ms delay - every bit adds up, and timers are much worse than event handling on e.g. mobile devices than servers, because they incur a CPU wake-up and then end up doing nothing most of the time.
Jun 21 2016