www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to setup D language with Apache httpd cgi?

reply BoQsc <vaidas.boqsc gmail.com> writes:
Hello, I would like to know how to setup D language project, so 
that it would work with Apache httpd cgi.

Do I need some kind of cgi library for D language? Right now all 
I'm getting is internal server error.

Btw. Unix Bash script examples seems to work well with with 
Apache httpd cgi, they do not show internal server error.


Before trying examples below, it is required that you install 
apache httpd:

 sudo apt-get install apache2


And enable cgi module

 sudo a2enmod cgi
 sudo service apache2 restart


Here are examples:

D language example, this one does not work. Give internal error.

/usr/lib/cgi-bin/example.d


import std.stdio;
void main()
{
    writeln("Hello, world with automated script running!");
}



Bash example, this one working perfectly.

/usr/lib/cgi-bin/example.sh


echo "Content-type: text/html"
echo ''
echo 'CGI Bash Example'



Also remember to chmod the examples, before opening them:

 sudo chmod 755 /usr/lib/cgi-bin/example.sh
 sudo chmod 755 /usr/lib/cgi-bin/example.d
Jul 30 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 30 July 2019 at 20:36:18 UTC, BoQsc wrote:

    writeln("Hello, world with automated script running!");




You didn't follow the cgi protocol here - you didn't output the 
content type and the required blank line to separate headers from 
content.


echo "Content-type: text/html"
echo ''
echo 'CGI Bash Example'




see how they did it here?


there are cgi libraries for D too (like my own) but for this 
simple case you just forgot a couple lines.
Jul 30 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Tuesday, 30 July 2019 at 21:55:00 UTC, Adam D. Ruppe wrote:

 the required blank line to separate headers from content.


That's exactly what causes the Internal Server Error.

This is a working example.d


import std.stdio;
void main()
{

    writeln("");


}


And this one, even produce HTML content to the HTML body tag:


import std.stdio;
void main()
{
    writeln("Content-type: text/html");
    writeln("");
    writeln("CGI D Example");

}


Thanks, Adam D. Ruppe, it works.
Jul 30 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 05:56:46 UTC, BoQsc wrote:

 what causes the Internal Server Error.


Internal Server Error might as well appear when D language syntax 
is not correct.

Considering this: this example has Internal Server Error, since 
writeln argument and argument content cannot contain the same 
type quotes: Internal Server Error


import std.stdio;
void main()
{
    writeln("Content-type: text/html");
    writeln("");
    writeln("<body style="background: green">CGI D 
 Example</body>");

}


This can be solved by using single quotes in the argument content 
places


import std.stdio;
void main()
{
    writeln("Content-type: text/html");
    writeln("");
    writeln("<body style='background: green'>CGI D 
 Example</body>");

}



Or even escaping argument content quotes:


import std.stdio;
void main()
{
    writeln("Content-type: text/html");
    writeln("");
    writeln("<body style=\"background: green\">CGI D 
 Example</body>");

}


That is good to know.
Jul 30 2019
next sibling parent reply 0xEAB <desisma heidel.beer> writes:
On Wednesday, 31 July 2019 at 06:30:03 UTC, BoQsc wrote:

 This can be solved by using single quotes in the argument 
 content places


import std.stdio;
void main()
{
    writeln("Content-type: text/html");
    writeln("");
    writeln("<body style='background: green'>CGI D 
 Example</body>");

}




Does the job but is a bad fix.

Use `` quotes for the string literal instead.
Further info: https://dlang.org/spec/lex.html#wysiwyg


  - Elias
Jul 30 2019
parent BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 06:52:46 UTC, 0xEAB wrote:

 On Wednesday, 31 July 2019 at 06:30:03 UTC, BoQsc wrote:

 This can be solved by using single quotes in the argument 
 content places


import std.stdio;
void main()
{
    writeln("Content-type: text/html");
    writeln("");
    writeln("<body style='background: green'>CGI D 
 Example</body>");

}




 Does the job but is a bad fix.

 Use `` quotes for the string literal instead.
 Further info: https://dlang.org/spec/lex.html#wysiwyg


  - Elias


I wasn't aware of Wysiwyg Strings, thanks,
seems to work very well.



import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);

}
Jul 31 2019
prev sibling parent reply 0xEAB <desisma heidel.beer> writes:
On Wednesday, 31 July 2019 at 06:30:03 UTC, BoQsc wrote:

 On Wednesday, 31 July 2019 at 05:56:46 UTC, BoQsc wrote:

 what causes the Internal Server Error.


 Internal Server Error might as well appear when D language 
 syntax is not correct.


Maybe you want to use some wrapper around rdmd that does output a 
proper CGI response on error. Since this error occurs because 
rdmd's error message as-is is not valid for CGI.


  - Elias
Jul 30 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 06:55:06 UTC, 0xEAB wrote:

 On Wednesday, 31 July 2019 at 06:30:03 UTC, BoQsc wrote:

 On Wednesday, 31 July 2019 at 05:56:46 UTC, BoQsc wrote:

 what causes the Internal Server Error.


 Internal Server Error might as well appear when D language 
 syntax is not correct.


 Maybe you want to use some wrapper around rdmd that does output 
 a proper CGI response on error. Since this error occurs because 
 rdmd's error message as-is is not valid for CGI.


  - Elias


I'm kind of new to everything, I'm not sure how this have to be 
done.
Jul 31 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
There are some other bad news, I switched from rdmd to dub 
package manager since I need a package from Adam D. Ruppe.

It is all good and well: this one works perfectly.


/+ dub.sdl:
	name "hello"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);


}


But, once I add dub dependency, the error appears: Internal 
Server Error


/+ dub.sdl:
	name "hello"
 	dependency "arsd-official" version="~>4.0.1"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);


}

This seems to work well when running not from cgi, so there is no 
syntax error.
Jul 31 2019
next sibling parent BoQsc <vaidas.boqsc gmail.com> writes:
I tried to change shebang, but: Internal Server Error still 
appears.



/+ dub.sdl:
	name "hello"
	dependency "arsd-official" version="~>4.0.1"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);

}
Jul 31 2019
prev sibling next sibling parent reply BoQsc <vaidas.boqsc gmail.com> writes:
 This seems to work well when running not from cgi, so there is


That was not true, it didn't work even from Linux Shell, I 
corrected shebang, now it works from Linux Shell.

An, unexpected thing: It did require permissions: sudo 
./example.d  which was not the case with rdmd.



cgi still shows Internal Server Error.

Maybe cgi cannot run this script due to lacking sudo permissions 
required by dub? Unsure.




/+ dub.sdl:
	name "hello"
	dependency "arsd-official" version="~>4.0.1"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);

}
Jul 31 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 07:42:22 UTC, BoQsc wrote:

 This seems to work well when running not from cgi, so there is


 That was not true, it didn't work even from Linux Shell, I 
 corrected shebang, now it works from Linux Shell.

 An, unexpected thing: It did require permissions: sudo 
 ./example.d  which was not the case with rdmd.



 cgi still shows Internal Server Error.

 Maybe cgi cannot run this script due to lacking sudo 
 permissions required by dub? Unsure.




/+ dub.sdl:
	name "hello"
	dependency "arsd-official" version="~>4.0.1"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);

}





It seems that /var/log/apache2/error.log shows Permission denied 
error.


[Wed Jul 31 10:44:26.887024 2019] [cgid:error] [pid 846:tid 
140090256426752] [client >127.0.0.1:57052] malformed header from 
script 'example.d': Bad header: Fetching arsd-official 4.0.1 (
/usr/lib/cgi-bin/.dub/packages/: Permission denied
Jul 31 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 07:57:01 UTC, BoQsc wrote:

[Wed Jul 31 10:44:26.887024 2019] [cgid:error] [pid 846:tid 
140090256426752] [client >127.0.0.1:57052] malformed header 
from script 'example.d': Bad header: Fetching arsd-official 
4.0.1 (
/usr/lib/cgi-bin/.dub/packages/: Permission denied




Added sudo to the shebang, it started to say that sudo requires 
tty, meaning that there is no graphical interface for user to 
input the password.


[Wed Jul 31 11:14:53.183607 2019] [cgid:error] [pid 6207:tid 
140374439274240] [client >127.0.0.1:57400] End of script output 
before headers: example.d
sudo: no tty present and no askpass program specified


Script:


/+ dub.sdl:
	name "hello"
	dependency "arsd-official" version="~>4.0.1"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);

}
Jul 31 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 08:32:43 UTC, BoQsc wrote:

 Added sudo to the shebang, it started to say that sudo requires 
 tty, meaning that there is no graphical interface for user to 
 input the password.


So, I remove the need to type password when using sudo command, 
by following these instructions:
https://www.cyberciti.biz/faq/linux-unix-running-sudo-command-without-a-password/

And this is the error I get now:

[Wed Jul 31 11:51:15.341790 2019] [cgid:error] [pid 870:tid 
140153708345088] [client >127.0.0.1:50318] End of script output 
before headers: example.d
Jul 31 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 09:03:47 UTC, BoQsc wrote:

 And this is the error I get now:

[Wed Jul 31 11:51:15.341790 2019] [cgid:error] [pid 870:tid 
140153708345088] [client >127.0.0.1:50318] End of script output 
before headers: example.d




And I have no idea how to deal with it. Cgi shows Internal Server 
Error, because of that "End of script output before headers" 
error.




/+ dub.sdl:
	name "hello"
	dependency "arsd-official" version="~>4.0.1"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);

}


Maybe it is not possible to use dub this way.
Jul 31 2019
next sibling parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 09:09:12 UTC, BoQsc wrote:

 On Wednesday, 31 July 2019 at 09:03:47 UTC, BoQsc wrote:

 And this is the error I get now:

[Wed Jul 31 11:51:15.341790 2019] [cgid:error] [pid 870:tid 
140153708345088] [client >127.0.0.1:50318] End of script 
output before headers: example.d




 And I have no idea how to deal with it. Cgi shows Internal 
 Server Error, because of that "End of script output before 
 headers" error.



It seems like apache does not like what dub package manager is 
doing, as dub does something before cgi script outputs anything.
https://stackoverflow.com/questions/22307610/end-of-script-output-before-headers-error-in-apache
Jul 31 2019
next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 31 July 2019 at 10:24:38 UTC, BoQsc wrote:

 On Wednesday, 31 July 2019 at 09:09:12 UTC, BoQsc wrote:

 On Wednesday, 31 July 2019 at 09:03:47 UTC, BoQsc wrote:

 And this is the error I get now:

[Wed Jul 31 11:51:15.341790 2019] [cgid:error] [pid 870:tid 
140153708345088] [client >127.0.0.1:50318] End of script 
output before headers: example.d




 And I have no idea how to deal with it. Cgi shows Internal 
 Server Error, because of that "End of script output before 
 headers" error.



 It seems like apache does not like what dub package manager is 
 doing, as dub does something before cgi script outputs anything.
 https://stackoverflow.com/questions/22307610/end-of-script-output-before-headers-error-in-apache


As info, the command
   dub app.d is
rewritten to:
   dub run -q --temp-build --single app.d

I wonder whether -q is correct here
   -q Only print warnings and errors
maybe --vquiet should be used instead
   -vquiet Print no messages

If there are any warnings, from dub / dmd these could cause 
invalid
Internal Server Errors.

Kind regards
André
Jul 31 2019
parent BoQsc <vaidas.boqsc gmail.com> writes:
Hmm, it seems that once I remove sudo from the shebang, the error 
disappears and Internal Server Error disappears.

example.d - does not work


	name "hello"
	
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);

}



/var/log/apache2/error.log


[Wed Jul 31 15:48:07.769766 2019] [cgid:error] [pid 4012:tid 
140324198397696] [client >127.0.0.1:51736] End of script output 
before headers: example.d



example.d - Works perfectly, no errors in the log


	name "hello"
	
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);

}
Jul 31 2019
prev sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 31 July 2019 at 10:24:38 UTC, BoQsc wrote:

 On Wednesday, 31 July 2019 at 09:09:12 UTC, BoQsc wrote:

 On Wednesday, 31 July 2019 at 09:03:47 UTC, BoQsc wrote:

 And this is the error I get now:

[Wed Jul 31 11:51:15.341790 2019] [cgid:error] [pid 870:tid 
140153708345088] [client >127.0.0.1:50318] End of script 
output before headers: example.d




 And I have no idea how to deal with it. Cgi shows Internal 
 Server Error, because of that "End of script output before 
 headers" error.



 It seems like apache does not like what dub package manager is 
 doing, as dub does something before cgi script outputs anything.
 https://stackoverflow.com/questions/22307610/end-of-script-output-before-headers-error-in-apache


Please see
https://github.com/adamdruppe/arsd/issues/203

Kind regards
André
Jul 31 2019
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 31 July 2019 at 09:09:12 UTC, BoQsc wrote:

	dependency "arsd-official" version="~>4.0.1"




I just changed the thing, so now you will want to use version 
4.0.2 and also require the cgi configuration rather than the 
default (I don't know how to do that in dub).

Though personally, when I do cgi stuff in D, I don't bother with 
rdmd or dub or whatever, I just compile the program separately 
and copy the binary in to the cgi directory. It is more efficient 
and simpler to handle in error cases.
Jul 31 2019
parent Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 31 July 2019 at 12:51:09 UTC, Adam D. Ruppe wrote:

 On Wednesday, 31 July 2019 at 09:09:12 UTC, BoQsc wrote:

	dependency "arsd-official" version="~>4.0.1"




 I just changed the thing, so now you will want to use version 
 4.0.2 and also require the cgi configuration rather than the 
 default (I don't know how to do that in dub).

 Though personally, when I do cgi stuff in D, I don't bother 
 with rdmd or dub or whatever, I just compile the program 
 separately and copy the binary in to the cgi directory. It is 
 more efficient and simpler to handle in error cases.


/+ dub.sdl:
     name "application"
     dependency "arsd-official:cgi" version="4.0.2"
     subConfiguration "arsd-official:cgi" "cgi"
+/

Kind regards
Andre
Jul 31 2019
prev sibling parent reply bachmeier <no spam.net> writes:
On Wednesday, 31 July 2019 at 07:19:12 UTC, BoQsc wrote:

 There are some other bad news, I switched from rdmd to dub 
 package manager since I need a package from Adam D. Ruppe.

 It is all good and well: this one works perfectly.


/+ dub.sdl:
	name "hello"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);


 }


 But, once I add dub dependency, the error appears: Internal 
 Server Error


/+ dub.sdl:
	name "hello"
 	dependency "arsd-official" version="~>4.0.1"
	
+/
import std.stdio;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`<body style="background: lightgreen">CGI D 
 Example</body>`);


 }

 This seems to work well when running not from cgi, so there is 
 no syntax error.


I'm curious why you don't just compile the binary and put it in 
the CGI directory. That's what I've always done. This AFAICT (I 
don't often use Dub) is going to compile every time it runs, 
which makes the program unnecessarily slow, and if it's used 
heavily, will add quite a load to the server. If you use Adam's 
cgi.d, you can test your program at the command line, and even 
run a server on localhost to test it in your browser.
Jul 31 2019
next sibling parent reply BoQsc <vaidas.boqsc gmail.com> writes:
 is going to compile every time it runs, which makes the program 
 unnecessarily slow, and if it's used heavily, will add quite a 
 load to the server.


I finally done it, and I'm not sure if it compiles every time. It 
opens the page lightning fast since I use SSD drive. I'm not sure 
how to check if it compiles every time.

Right now, I'm logged in as root and chmoded recursively the 
whole /usr/lib/cgi-bin/ folder
and it succeed.




    name "application"
    dependency "arsd-official:dom" version="4.0.2"
    subConfiguration "arsd-official:cgi" "cgi"
+/
import std.stdio;
import arsd.dom;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
 
    auto document = new 
 Document("<html><body><p>paragraph</p></body></html>");
    document.root.innerHTML = "<p>hey</p>";
    writeln(document);

//    writeln(`<body style="background: lightgreen">CGI D 
Example</body>`);

}
Jul 31 2019
parent reply BoQsc <vaidas.boqsc gmail.com> writes:
However I tried to add options (--single) to the dub shebang and 
apache now throwing: "bad header error"

Without "--single" option seems to work.




/+ dub.sdl:
    name "application"
    dependency "arsd-official:dom" version="4.0.2"
+/
import std.stdio;
import arsd.dom;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`test`);

}





split -S:  'dub --single'
 into:    'dub'
     &    '--single'
executing: dub
   arg[0]= 'dub'
   arg[1]= '--single'
   arg[2]= '/usr/lib/cgi-bin/example.d'
[Wed Jul 31 19:57:56.892522 2019] [cgid:error] [pid 833:tid 
140583783876352] [client >127.0.0.1:43598] malformed header from 
script 'example.d': Bad header: Performing "debug" >build using
Jul 31 2019
parent Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 31 July 2019 at 17:04:58 UTC, BoQsc wrote:

 However I tried to add options (--single) to the dub shebang 
 and apache now throwing: "bad header error"

 Without "--single" option seems to work.




/+ dub.sdl:
    name "application"
    dependency "arsd-official:dom" version="4.0.2"
+/
import std.stdio;
import arsd.dom;
void main()
{
    writeln(`Content-type: text/html`);
    writeln(``);
    writeln(`test`);

}





split -S:  'dub --single'
 into:    'dub'
     &    '--single'
executing: dub
   arg[0]= 'dub'
   arg[1]= '--single'
   arg[2]= '/usr/lib/cgi-bin/example.d'
[Wed Jul 31 19:57:56.892522 2019] [cgid:error] [pid 833:tid 
140583783876352] [client >127.0.0.1:43598] malformed header 
from script 'example.d': Bad header: Performing "debug" >build 
using




As already pointed out, as starting point you could compile the 
application e.g. within folder /tmp and then copy the executable 
to cgi-bin folder.
Please make sure it has executable flag.

dub build --single hello.d

The application could look like this:


/+ dub.sdl:
     name "application"
     dependency "arsd-official:cgi" version="4.0.2"
     subConfiguration "arsd-official:cgi" "cgi"
+/
import arsd.cgi;

void hello(Cgi cgi) {                                             
                      cgi.setResponseContentType("text/plain");

     if("name" in cgi.get)
         cgi.write("Hello, " ~ cgi.get["name"]);
     else
         cgi.write("Hello, world!");
  }

mixin GenericMain!hello;

Kind regards
Andre
Jul 31 2019
prev sibling parent reply BoQsc <vaidas.boqsc gmail.com> writes:
On Wednesday, 31 July 2019 at 14:19:20 UTC, bachmeier wrote:


 is going to compile every time it runs, which makes the program 
 unnecessarily slow


If only I could add

dub --single --rdmd


to the shebang, I think dub might stop compiling every time.

However as pointed out in the above post, I'm unable to use 
options in shebang since apache is throwing header error for 
unknown reasons.
Jul 31 2019
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jul 31, 2019 at 05:09:50PM +0000, BoQsc via Digitalmars-d-learn wrote:
[...]

 However as pointed out in the above post, I'm unable to use options in
 shebang since apache is throwing header error for unknown reasons.


[...]

Usually, that's a sign that somebody is writing to stdout/stderr that
should not be, because apache usually redirects stdout to the actual
HTTP output stream.  So if dub writes to stdout from within an apache
module, that will garble the HTTP output stream and cause header errors
and other kinds of errors.  You may need to somehow redirect
stdout/stderr if this is the case.


T

-- 
We've all heard that a million monkeys banging on a million typewriters will
eventually reproduce the entire works of Shakespeare.  Now, thanks to the
Internet, we know this is not true. -- Robert Wilensk
Jul 31 2019