www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A web server with D

reply Mengu <mengukagan gmail.com> writes:
Hi,

I have been interested in and learning D for a while and currently developing
a web development IDE with it. I can say that I have a middle level knowledge
that I have been trying to increase. Anyway. I want to develop a web server
for Python and Ruby web applications some months later from now,
after completing the IDE. But until then I want all my research to be
completed and I have enough knowledge that I need to develop this web server,
in addition to D and within D.

So, my two questions are these:

  1) What do I need to know in order to develop a web server, generally?
  2) What do I need to know within D to make this good enough?

Thanks everyone in advance.
Jun 20 2010
next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 20/06/10 12:54, Mengu wrote:
 Hi,

 I have been interested in and learning D for a while and currently developing
 a web development IDE with it. I can say that I have a middle level knowledge
 that I have been trying to increase. Anyway. I want to develop a web server
 for Python and Ruby web applications some months later from now,
 after completing the IDE. But until then I want all my research to be
 completed and I have enough knowledge that I need to develop this web server,
 in addition to D and within D.

 So, my two questions are these:

    1) What do I need to know in order to develop a web server, generally?

A good starting point would be taking a look at current web servers and seeing what they do well etc. Apache httpd is the most widely used one which would be a good starting point, particularly feature wise, but it's also worth taking a look at others such as lighttpd, nginx and cherokee. Note that the main selling points for the latter are speed rather than functionality (although I believe this is changing). From doing quite a bit of server sided development myself I'd also recommend reading the HTTP RFC[1] thoroughly, a good few times and making sure you understand it completely. Understand its downfalls and intricacies, places where you could encounter issues, make sure you've thought about how you're gonna get around them before you start. A lot of web servers have already got them covered, so again taking a look at them would be a good starting point. It would also be worth taking a look at best practices server-wise. I can't think of any off the top of my head, I know there are some things which are better done one way than another - you should take a look at these before you start too. As an afterthought, it could be a good idea to take a look at security issues that have been reported for other httpd's, and making sure such vulnerabilities don't exist in your server in the first place! There will be some that are very httpd specific, I have no doubt there will be others that crop up from time to time relating to similar parts of the httpd.
    2) What do I need to know within D to make this good enough?

Why settle with good enough when you can settle for great? I don't think there's anything you specifically need to know within D for making a webserver, but you should make sure you know your way around the concurrency/threading code and also the sockets interface so you can make good use of them. Concurrency support continues to evolve in D, so it'd be worth keeping an eye on.
 Thanks everyone in advance.

Thanks, Robert [1] http://www.w3.org/Protocols/rfc2616/rfc2616.html - yes, I know it's a bit of a monster read, if you're gonna do this you need to do it properly though :)
Jun 20 2010
parent sybrandy <sybrandy gmail.com> writes:
 1) What do I need to know in order to develop a web server, generally?

A good starting point would be taking a look at current web servers and seeing what they do well etc. Apache httpd is the most widely used one which would be a good starting point, particularly feature wise, but it's also worth taking a look at others such as lighttpd, nginx and cherokee. Note that the main selling points for the latter are speed rather than functionality (although I believe this is changing).

I haven't looked at it too deeply, but Mongrel2 looks to be very interesting. IIRC, it's written in pure C and supports several different protocols. I believe it leans heavily on the performance side. However, it's not finished yet. Since D's concurrency is going to use message passing, you may want to check out Yaws which is written in Erlang to see how they handle connections and what not. However, do keep in mind that creating threads in D is much more expensive than creating processes in Erlang. Casey
Jun 20 2010
prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
Mengu Wrote:

 Hi,
 
 I have been interested in and learning D for a while and currently developing
 a web development IDE with it. I can say that I have a middle level knowledge
 that I have been trying to increase. Anyway. I want to develop a web server
 for Python and Ruby web applications some months later from now,
 after completing the IDE. But until then I want all my research to be
 completed and I have enough knowledge that I need to develop this web server,
 in addition to D and within D.
 
 So, my two questions are these:
 
   1) What do I need to know in order to develop a web server, generally?
   2) What do I need to know within D to make this good enough?

Read the HTTP RFC, tear your hair out as you realize that the grammar isn't context-free, and console yourself with the knowledge that it's still one of the most solid and well-followed internet protocol RFCs. There are a few RFCs on cookies you'll need to look at too. After that, if you want a high-performance server I'd suggest looking into libevent.
Jun 20 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Sean Kelly" <sean invisibleduck.org> wrote in message 
news:hvlf5u$18ae$1 digitalmars.com...
 Mengu Wrote:

 Hi,

 I have been interested in and learning D for a while and currently 
 developing
 a web development IDE with it. I can say that I have a middle level 
 knowledge
 that I have been trying to increase. Anyway. I want to develop a web 
 server
 for Python and Ruby web applications some months later from now,
 after completing the IDE. But until then I want all my research to be
 completed and I have enough knowledge that I need to develop this web 
 server,
 in addition to D and within D.

 So, my two questions are these:

   1) What do I need to know in order to develop a web server, generally?
   2) What do I need to know within D to make this good enough?

Read the HTTP RFC, tear your hair out as you realize that the grammar isn't context-free

Really? It seems so simple (but of course, XML seems really simple at first glance, too). Have an example?
Jun 20 2010
parent Sean Kelly <sean invisibleduck.org> writes:
Nick Sabalausky Wrote:

 "Sean Kelly" <sean invisibleduck.org> wrote in message 
 Read the HTTP RFC, tear your hair out as you realize that the grammar 
 isn't context-free

Really? It seems so simple (but of course, XML seems really simple at first glance, too). Have an example?

I may have exaggerated a tad, but the issue I was thinking of is that the header values can contain pretty much anything--particularly cookies (no one follows the cookie RFC). So it's not uncommon to see: Cookie: myval=thing1:thing2 And then of course the body is just a blob of whatever. So you can't just tokenize an HTTP request the same way you would a D module, for example. You have to know whether you're in the control line (GET /thingy HTTP/1.1), header key, header value, or body part. What I've found so far is that it's easiest to simply break a request into control line, set of header key/value pairs (of which there can be duplicates of the same key), and body blob and then parse individual headers or the body separately as needed. This is more efficient anyway I suppose. It's just irksome that it seems necessary.
Jun 20 2010
prev sibling parent "Rory McGuire" <rmcguire neonova.co.za> writes:
On Sun, 20 Jun 2010 13:54:26 +0200, Mengu <mengukagan gmail.com> wrote:

 Hi,

 I have been interested in and learning D for a while and currently  
 developing
 a web development IDE with it. I can say that I have a middle level  
 knowledge
 that I have been trying to increase. Anyway. I want to develop a web  
 server
 for Python and Ruby web applications some months later from now,
 after completing the IDE. But until then I want all my research to be
 completed and I have enough knowledge that I need to develop this web  
 server,
 in addition to D and within D.

 So, my two questions are these:

   1) What do I need to know in order to develop a web server, generally?
   2) What do I need to know within D to make this good enough?

 Thanks everyone in advance.

read the web server related RFCs, not just HTTP (e.g. also MIME). try to keep everthing re-usable, e.g. make a library and a web server. make sure you allow re-writing of urls according to rules (see apache mod rewrite) and: SSL, Virtual hosts (both named and IP based) (ssl doesn't support name based virtual hosts), access control. Logging is also very important.
Jun 21 2010