www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - D daemon & GC?

reply "JD" <jd jd.com> writes:
Hi all,

I tried to write a Linux daemon in D 2.065 (by translating one  
in C we use at work). My basic skeleton works well. But as soon 
as I start allocating memory it crashed with several 
'core.exception.InvalidMemoryOperationError's.

My questions:
1. Are there any special considerations w.r.t. the GC after using 
fork()? Or must it be disabled?
2. Is it allowed to use stdlib's exit() without cleaning up after 
the runtime (as a normal end of program probably does)? Or is 
there a safe D exit()?

I could not find any working daemon examples, so any help is 
appreciated!

Bye,
Jeroen

---

My code so far:

module testdaemon;

import std.stdio;
import std.c.stdlib : exit, EXIT_FAILURE, EXIT_SUCCESS;
import core.thread : Thread, dur;
import core.sys.posix.sys.stat : umask;
import core.sys.posix.unistd : fork, setsid, chdir, close, 
STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO;

void daemonize()
{
	// fork this process
	auto pid = fork();
	if (pid == -1) exit(EXIT_FAILURE);

	// this is the parent; terminate it
	if (pid > 0)
	{
		writefln("Starting daemon mode, process id = %d\n", pid);
		// is this ok, or should we clean up after the runtime?
		exit(EXIT_SUCCESS);
	}
	
	//unmask the file mode
	umask(0);
	
	// become process group leader
	auto sid = setsid();
	if(sid < 0) exit(EXIT_FAILURE);
	
	// do not lock any directories
	chdir("/");

	// Close stdin, stdout and stderr
	close(STDIN_FILENO);
	close(STDOUT_FILENO);
	close(STDERR_FILENO);
}

void main()
{	
	auto log = File("testdaemon.log", "w");
	daemonize();
	while(true)
	{	
		try
		{	
			// this statement causes 
core.exception.InvalidMemoryOperationError
			// auto t = new char[4096];

			log.write(".");
		}
		catch(Exception e)
		{
			log.write(e.msg);
		}
		finally
		{
			log.flush();
		}
		Thread.sleep(dur!("seconds")(5));
	}
}
Aug 30 2014
next sibling parent reply "safety0ff" <safety0ff.dev gmail.com> writes:
On Saturday, 30 August 2014 at 17:09:41 UTC, JD wrote:
 Hi all,

 I tried to write a Linux daemon in D 2.065 (by translating one  
 in C we use at work). My basic skeleton works well. But as soon 
 as I start allocating memory it crashed with several 
 'core.exception.InvalidMemoryOperationError's.
It works for me with 2.066, I do not have 2.065 installed at the moment to see if it fails on 2.065.
Aug 30 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 30 August 2014 at 17:36:41 UTC, safety0ff wrote:
 On Saturday, 30 August 2014 at 17:09:41 UTC, JD wrote:
 Hi all,

 I tried to write a Linux daemon in D 2.065 (by translating one
  in C we use at work). My basic skeleton works well. But as 
 soon as I start allocating memory it crashed with several 
 'core.exception.InvalidMemoryOperationError's.
It works for me with 2.066, I do not have 2.065 installed at the moment to see if it fails on 2.065.
I tried to test it with Digger (in reverse mode), and it complained that the test succeeded with 2.065 (the supposedly bad version), so I guess I can't reproduce it with 2.065 here either.
Aug 30 2014
parent reply "JD" <jd jd.com> writes:
Oops, I accidentally commented out the line allocating the memory 
in the example code... sorry.

	// this statement causes 
core.exception.InvalidMemoryOperationError
	// auto t = new char[4096];

should read:

	// this statement causes 
core.exception.InvalidMemoryOperationError
	auto t = new char[4096];

I can confirm that the daemon is not running with dmd 2.065:

$ ./testdaemon
Starting daemon mode, process id = 30147

$ ps -ef | grep testdaemon
   501 30149 35108   0  8:17pm ttys009    0:00.00 grep testdaemon

By removing the "close(STDOUT_FILENO);" from the daemon function, 
the InvalidMemoryOperationErrors appear on the screen.


On Saturday, 30 August 2014 at 18:05:01 UTC, Marc Sch├╝tz wrote:
 On Saturday, 30 August 2014 at 17:36:41 UTC, safety0ff wrote:
 On Saturday, 30 August 2014 at 17:09:41 UTC, JD wrote:
 Hi all,

 I tried to write a Linux daemon in D 2.065 (by translating one
 in C we use at work). My basic skeleton works well. But as 
 soon as I start allocating memory it crashed with several 
 'core.exception.InvalidMemoryOperationError's.
It works for me with 2.066, I do not have 2.065 installed at the moment to see if it fails on 2.065.
I tried to test it with Digger (in reverse mode), and it complained that the test succeeded with 2.065 (the supposedly bad version), so I guess I can't reproduce it with 2.065 here either.
Aug 30 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 30 August 2014 at 18:22:49 UTC, JD wrote:
 Oops, I accidentally commented out the line allocating the 
 memory in the example code... sorry.

 	// this statement causes 
 core.exception.InvalidMemoryOperationError
 	// auto t = new char[4096];

 should read:

 	// this statement causes 
 core.exception.InvalidMemoryOperationError
 	auto t = new char[4096];
Ah, of course. I could have seen that myself :-P I can already say that it nevertheless works with DMD git. Will test soon with Digger, unfortunately Bitbucket is currently down, and Digger depends on it.
Aug 30 2014
parent reply "JD" <jd jd.com> writes:
 I can already say that it nevertheless works with DMD git. Will 
 test soon with Digger, unfortunately Bitbucket is currently 
 down, and Digger depends on it.
In the meantime I installed DMD 2.066 and I changed the exit() function after the fork as Dicebot suggested. Unfortunately I got the same result: $ ./testdaemon Starting daemon mode, process id = 30510 core.exception.InvalidMemoryOperationError (0) core.exception.InvalidMemoryOperationError (0) I think the issue is the way the parent is stopped after the fork(). Being quite new to D I have to familiarise myself some more with D's runtime and GC to see it... --- The changed example code: module testdaemon; import std.stdio; import std.c.stdlib : exit, EXIT_FAILURE, EXIT_SUCCESS; import core.thread : Thread, dur; import core.sys.posix.sys.stat : umask; import core.sys.posix.unistd : fork, setsid, chdir, close, STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO; extern (C) { void _Exit(int); } void daemonize() { // fork this process auto pid = fork(); if (pid == -1) exit(EXIT_FAILURE); // this is the parent; terminate it if (pid > 0) { writefln("Starting daemon mode, process id = %d\n", pid); _Exit(EXIT_SUCCESS); } //unmask the file mode umask(0); // become process group leader auto sid = setsid(); if(sid < 0) exit(EXIT_FAILURE); // do not lock any directories chdir("/"); // Close stdin, stdout and stderr close(STDIN_FILENO); // close(STDOUT_FILENO); // close(STDERR_FILENO); } void main() { auto log = File("testdaemon.log", "w"); daemonize(); while(true) { try { // this statement causes core.exception.InvalidMemoryOperationError auto t = new char[4096]; log.write("."); } catch(Exception e) { log.write(e.msg); } finally { log.flush(); } Thread.sleep(dur!("seconds")(5)); } }
Aug 30 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 30 August 2014 at 19:20:39 UTC, JD wrote:
 I can already say that it nevertheless works with DMD git. 
 Will test soon with Digger, unfortunately Bitbucket is 
 currently down, and Digger depends on it.
In the meantime I installed DMD 2.066 and I changed the exit() function after the fork as Dicebot suggested. Unfortunately I got the same result:
I tested it again, and it works fine in both 2.065 and 2.066. At least theoretically, whatever happens in the parent should not affect the child, so I don't really believe it has something to do with the way exit() works. You can actually test this by making the parent not exit immediately, but sleep for some time. And just to be sure: have you tested your program without the call to daemonize?
Aug 30 2014
parent reply "JD" <jd jd.com> writes:
 I tested it again, and it works fine in both 2.065 and 2.066.
Be aware that you should comment out: // close(STDOUT_FILENO); // close(STDERR_FILENO); A daemon normally detaches itself from the terminal by closing the STD* files. All D's runtime exception messages will not appear in that case.
 At least theoretically, whatever happens in the parent should 
 not affect the child, so I don't really believe it has 
 something to do with the way exit() works. You can actually 
 test this by making the parent not exit immediately, but sleep 
 for some time.
Good point. I'll try that.
 And just to be sure: have you tested your program without the 
 call to daemonize?
Yes, it writes a dot to the logfile every 2 seconds :-)
Aug 30 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Saturday, 30 August 2014 at 19:52:24 UTC, JD wrote:
 I tested it again, and it works fine in both 2.065 and 2.066.
Be aware that you should comment out: // close(STDOUT_FILENO); // close(STDERR_FILENO); A daemon normally detaches itself from the terminal by closing the STD* files. All D's runtime exception messages will not appear in that case.
 At least theoretically, whatever happens in the parent should 
 not affect the child, so I don't really believe it has 
 something to do with the way exit() works. You can actually 
 test this by making the parent not exit immediately, but sleep 
 for some time.
Good point. I'll try that.
 And just to be sure: have you tested your program without the 
 call to daemonize?
Yes, it writes a dot to the logfile every 2 seconds :-)
Last snippet works for me, dots get printed to the logfile as expected.
Aug 30 2014
parent reply "JD" <jd jd.com> writes:
 Last snippet works for me, dots get printed to the logfile as 
 expected.
Ok, it works now. Using the recommended _Exit() function with DMD 2.066 on Linux. Thanks you all for your help! Best regards, Jeroen
Aug 31 2014
parent "Gary Willoughby" <dev nomad.so> writes:
On Sunday, 31 August 2014 at 09:02:55 UTC, JD wrote:
 Last snippet works for me, dots get printed to the logfile as 
 expected.
Ok, it works now. Using the recommended _Exit() function with DMD 2.066 on Linux. Thanks you all for your help! Best regards, Jeroen
On a side note, i've created daemons like this before but then i found a rather nice posix system call to do it all for me: extern (C) { /** * The daemon() function is for programs wishing to detach themselves * from the controlling terminal and run in the background as system * daemons. * * (This function forks, and if the fork(2) succeeds, the parent calls * _exit(2), so that further errors are seen by the child only.) On * success daemon() returns zero. If an error occurs, daemon() returns * -1 and sets errno to any of the errors specified for the fork(2) and * setsid(2). * * Params: * nochdir = If nochdir is zero, daemon() changes the calling process's * current working directory to the root directory ("/"); otherwise, * the current working directory is left unchanged. * noclose = If noclose is zero, daemon() redirects standard input, * standard output and standard error to /dev/null; otherwise, no * changes are made to these file descriptors. */ int daemon(int nochdir, int noclose); } This is a lot easier to use. :)
Aug 31 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
I had similar issues. Thing is D compiler places runtime cleanup 
& initialization code into binary "constructor" sections 
(fini_array) which is needed to support shared libraries 
properly. But the same code gets triggered when you run `exit` in 
a fork resulting in attempt to terminate other process GC. There 
is an alternative "_Exit" function that should be used in forks 
instead. This is a link on topic given to me earlier by Martin 
Nowak : 
http://unix.stackexchange.com/questions/5364/why-should-a-child-of-a-vfork-or-fork-call-exit-instead-of-exit

I am not 100% this is what causes your problems but seems 
possible.
Aug 30 2014
prev sibling next sibling parent reply "Israel" <tl12000 live.com> writes:
On Saturday, 30 August 2014 at 17:09:41 UTC, JD wrote:
 Hi all,
 Or is there a safe D exit()?
I always wondered why D doesnt have this. I tried searching google and could not find a definitive answer except interfacing with C...
Aug 30 2014
parent ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Sat, 30 Aug 2014 19:20:44 +0000
Israel via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:

 Or is there a safe D exit()?
I always wondered why D doesnt have this.
the "idiomatic" way is to throw an exception which can be catched in main() (and then we can `return 1` from main() or something like it). or not catch anything and runtime will terminate the program.
Aug 30 2014
prev sibling parent =?UTF-8?B?UmFwaGHDq2wgSmFrc2U=?= <raphael.jakse gmail.com> writes:
On 30/08/2014 19:09, JD wrote:
 My questions:
 1. Are there any special considerations w.r.t. the GC after using
 fork()? Or must it be disabled?
 2. Is it allowed to use stdlib's exit() without cleaning up after the
 runtime (as a normal end of program probably does)? Or is there a safe D
 exit()?
For the second question, Runtime.terminate() from the core.runtime package might be a possible answer. http://dlang.org/phobos/core_runtime.html#.Runtime.terminate I don't know how calling Runtime.terminate() in forks behaves though.
Aug 31 2014