www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Web development status

reply "Antti Holvikari" <anttih gmail.com> writes:
Hi!

I've searched and searched for some stuff about web development in D
but have found almost nothing. What is the status of the CGI module[1]
that Marten wrote (the link on that page is broken)? Or should I go
with Mango?

Has anyone done any webapps with D?

[1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI

-- 
Antti Holvikari
Jan 21 2007
next sibling parent BCS <ao pathlink.com> writes:
Reply to Antti,

 Hi!
 
 I've searched and searched for some stuff about web development in D
 but have found almost nothing. What is the status of the CGI module[1]
 that Marten wrote (the link on that page is broken)? Or should I go
 with Mango?
 
 Has anyone done any webapps with D?
 
 [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
 
as for CGI: I have built a local search engine in D (search local finls, v1 was built with find/grep/sed/cat/etc.) http://www.dsource.org/projects/scrapple/browser/trunk/search http://www.dsource.org/projects/scrapple/browser/trunk/cgi and a bare bones NNTP viewer http://www.dsource.org/projects/scrapple/browser/trunk/nntp but IIRC that is missing some of the code (I'l try to rectify that if I can find the ret of that app.
Jan 21 2007
prev sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Antti Holvikari wrote:
 Hi!
 
 I've searched and searched for some stuff about web development in D
 but have found almost nothing. What is the status of the CGI module[1]
 that Marten wrote (the link on that page is broken)? Or should I go
 with Mango?
 
 Has anyone done any webapps with D?
 
 [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
 
I wish we'd have a framework similar to Django, however, come to think of it .. I'd rather use Django than wait for a D clone .. which I don't think can be more powerful since D doesn't have any dynamic reflection capabilities.
Jan 22 2007
parent reply "Antti Holvikari" <anttih gmail.com> writes:
On 1/22/07, Hasan Aljudy <hasan.aljudy gmail.com> wrote:
 Antti Holvikari wrote:
 Hi!

 I've searched and searched for some stuff about web development in D
 but have found almost nothing. What is the status of the CGI module[1]
 that Marten wrote (the link on that page is broken)? Or should I go
 with Mango?

 Has anyone done any webapps with D?

 [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
I wish we'd have a framework similar to Django, however, come to think of it .. I'd rather use Django than wait for a D clone .. which I don't think can be more powerful since D doesn't have any dynamic reflection capabilities.
Hmm, hope I'm not missing something here but why would you need reflection for that? -- Antti Holvikari <http://phphalo.com>
Jan 22 2007
next sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Antti Holvikari wrote:
 On 1/22/07, Hasan Aljudy <hasan.aljudy gmail.com> wrote:
 Antti Holvikari wrote:
 Hi!

 I've searched and searched for some stuff about web development in D
 but have found almost nothing. What is the status of the CGI module[1]
 that Marten wrote (the link on that page is broken)? Or should I go
 with Mango?

 Has anyone done any webapps with D?

 [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
I wish we'd have a framework similar to Django, however, come to think of it .. I'd rather use Django than wait for a D clone .. which I don't think can be more powerful since D doesn't have any dynamic reflection capabilities.
Hmm, hope I'm not missing something here but why would you need reflection for that?
Well, We don't need it to make a server in D, but if we want to make a powerful framework for web applications, I'd say reflection is a must. Maybe I don't understand Reflection very well, but there are things in Django that'd only be possible in a dynamic language, or a language with good Reflection capabilities.
Jan 22 2007
parent reply kenny <funisher gmail.com> writes:
Hasan Aljudy wrote:
 
 
 Antti Holvikari wrote:
 On 1/22/07, Hasan Aljudy <hasan.aljudy gmail.com> wrote:
 Antti Holvikari wrote:
 Hi!

 I've searched and searched for some stuff about web development in D
 but have found almost nothing. What is the status of the CGI module[1]
 that Marten wrote (the link on that page is broken)? Or should I go
 with Mango?

 Has anyone done any webapps with D?

 [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
I wish we'd have a framework similar to Django, however, come to think of it .. I'd rather use Django than wait for a D clone .. which I don't think can be more powerful since D doesn't have any dynamic reflection capabilities.
Hmm, hope I'm not missing something here but why would you need reflection for that?
Well, We don't need it to make a server in D, but if we want to make a powerful framework for web applications, I'd say reflection is a must. Maybe I don't understand Reflection very well, but there are things in Django that'd only be possible in a dynamic language, or a language with good Reflection capabilities.
I've been doing web development in D for a long time. I use my own framework to do it. If there are enough people interested, I could possibly open source it in a few weeks (it needs a lot of cleaning) it connects over fastcgi interface (I run lighttpd, but apache can be used), or can use shttpd's embedded for connections directly. It's quite extensive -- it's parsed templates into bytecode, and quite a bit of other stuff... It's definitely pretty solid, and it's the backend of a social network that's being built. if you guys are really interested, I'll write some samples up to get feedback before opening it.
Jan 22 2007
parent reply Alexander Panek <a.panek brainsware.org> writes:
kenny wrote:
 Hasan Aljudy wrote:
 Antti Holvikari wrote:
 On 1/22/07, Hasan Aljudy <hasan.aljudy gmail.com> wrote:
 Antti Holvikari wrote:
 Hi!

 I've searched and searched for some stuff about web development in D
 but have found almost nothing. What is the status of the CGI 
module[1]
 that Marten wrote (the link on that page is broken)? Or should I go
 with Mango?

 Has anyone done any webapps with D?

 [1] http://www.prowiki.org/wiki4d/wiki.cgi?CGI
I wish we'd have a framework similar to Django, however, come to think of it .. I'd rather use Django than wait for a D clone .. which I don't think can be more powerful since D doesn't have any dynamic reflection capabilities.
Hmm, hope I'm not missing something here but why would you need reflection for that?
Well, We don't need it to make a server in D, but if we want to make a powerful framework for web applications, I'd say reflection is a must. Maybe I don't understand Reflection very well, but there are things in Django that'd only be possible in a dynamic language, or a language with good Reflection capabilities.
I've been doing web development in D for a long time. I use my own framework to do it. If there are enough people interested, I could possibly open source it in a few weeks (it needs a lot of cleaning) it connects over fastcgi interface (I run lighttpd, but apache can be used), or can use shttpd's embedded for connections directly. It's quite extensive -- it's parsed templates into bytecode, and quite a bit of other stuff... It's definitely pretty solid, and it's the backend of a social network that's being built. if you guys are really interested, I'll write some samples up to get feedback before opening it.
That sounds amazing!! Examples would be great, though. :)
Jan 22 2007
next sibling parent kenny <funisher gmail.com> writes:
ok, here is some explanation of the framework I use: Sorry for the length of
the post. I promise I will never make one this big again. It took quite some
time to write.

1. most of the core is somewhat contract programmed and heavily unittested --
that is, I'm pretty sure it won't break, and the few times I've got a few good
ideas, the unittests told me they weren't so good.

2. connecting to the content server can be done one of two ways: 1. over
fastcgi (I use lighttpd) or 2. through shttpd embedded. Soon, a third may be
added, integrating it directly into the kernel -- similar to the red hat
content server, and I'm looking into every possible way to have D load up the
various modules in real time. (eg. call a function, and all of the .d files are
recompiled and reloaded) This could be done similar to derelict style, or maybe
through the mixins idea somewhere else in this newsgroup.

3. it's very fast -- yet has tons of opportunities to be faster (possibly
double the speed). Originally, I started with straight output using fastcgi's
print functions, and string concats. concatenation is very slow, so I moved to
a variadic function to print stuff out (cause templates weren't good enough
back then). I now use a template system for design, which parses the template
into "panels" then stores the byte code generated for later rendering. This
decreased performance by 50% -- but in the future, the D-templates could easily
parse it, (if it can ever accept flat files for input) and easily generate
machine code to do just that. The beauty of templates though, is templates can
change, and they won't be reloaded in memory until I push the button. Also, no
downtime either. In every example I had MySQL was ALWAYS the limiting factor.
On my laptop, if I cached the data in RAM, I could easily do 6000-14000
requests a second, but using mysql, that number dropped to about 1
000 
requests per second (for a dynamic page). Profiling shows that mysql takes
about 80%-90% of the time, so improving the speed of templates will yield
little results.

4. it's still pretty immature. I will need help getting it super stable and
production worthy. There are no bugs that I know of, but that doesn't mean they
don't exist.

5. I use custom links and forms, but they are not necessary. This is because
the program I'm writing needs everything to go through an iframe or through
ajax, so I don't mess with the layers on top. It gets complicated explaining,
so I just won't... the standard <a href="?lala=2">link</a> will work quite
fine. It's optional. Because of the nature of what I do, the applications must
start having more motion. So, one of the directions I'm moving in, is to make
it easier to write inline the stuff to mess with the javascript/dom.

6. you will see them... the qry(...) and the prt(...) are hideous... they
shouldn't look after a ^ to insert the data. It should be templated version of
writef that everything is made on compile time. Later that can be made... for
now, who cares... it's fast enough, and old code dies hard.

---------------

first, it searches for main.pnl in the panels dir and starts from there... here
are various template parameters that can be seen as examples... HTML is
designed to be completely intermingled

<%interface name: "myprofile" %>
<%load Base %>
<%load Me %>
<%if uid == 0 %>
	<b>you are not logged in</b>
<%else%>
	<div>your name is <%=Me.fullname%>
	<%load Users:Friends {friends_gid: $Me.friends_gid } %>
	<div>You have <%=Users.total%> friends</div>
	<%loop Users:Friends %>
	// copy and pasted from one of my projects...
             <div class="friend">
                 <div class="izquierda_amigos">
                     <img src="i/60/<%=Users.pic%>" alt="<%=Users.fullname%>" />
                 </div>
                 <div class="derecha_amigos">

<%=Users.lastname%></a></h3>
                 </div>
             </div>
            <%endloop%>
<%endif%>

you can make/set vars in the templates...
<%variable name: "varname" type: "uint" default: "2" %>
<%set name: "varname" value: 3 %>

linking is easy... in any panel, you can define where a sub panel will go:

// ex: main.pnl
<%interface name: "main" %>
html... blah blah blah
<%panel name: "m" default: "home" %>
more html blah blah blah

then in the url, to put the panel "home" inside of the subpanel location "m"
you do nothing... however, let's say that you want the panel named "myprofile"
in there. That's done with the URL string:

<a href="?m=myprofile">link to my profile</a>

pretty easy.

for those looking for a truly object oriented design, don't look here... lol,
it looks like I'm writing C/PHP mix most of the time, cause of my heavy use of
assoc arrays and loops for parsing...

The modules are written in D and are quite easy to make as well... There are
three types... Loopable, One-shot (no loops), and read only (for feedback to
forms) For example, let me show you what it takes to make the above
Users:Friends module (it's a loopable type):

static class Users : TemplateStaticImport {
	static:
	char[] firstname;
	char[] lastname;
	char[] fullname;
	char[] pic;
	char[][][] res;
	
	uint page;
	uint size;

	const char[] name = "Users";
	
	static this() {
		// all users in the website
		GenericLoop!(name ~ ":Site", generic_init,
			function () {
				reload = true;
				total = toUint(q1v("SELECT COUNT(*) FROM `users`"));
			},
			function int() {
				if(reload) {
					res = qry("SELECT `uid`, `firstname`, `lastname`, `fullname`, `pic` FROM
`users`");
					count = res.length;
				}
				
				return generic_loop;
			});
		
		// You can make as many of these as you want in this class actually...
technically, you don't even need it in a class, but it is for now here in a
class, because they used to be instantiated, not static...

		// just the group group of people I ask (which happen to be friends of the
person)
		GenericLoop!(name ~ ":Friends", generic_init,
			function () {
				assert(friends_gid);
				assert(*friends_gid);
				reload = true;
				total = toUint(q1v("SELECT COUNT(*) FROM `group_members` WHERE `type` = '"
~ s_gid_user_friends ~ "' AND `gid` = '^' AND `id` != '^'", *friends_gid, uid));
			},
			function int() {
				if(reload) {
					assert(friends_gid);
					assert(*friends_gid);
					// I know it's more optimized to do a direct relations table here, but
that's not what we need... we need generic groups of people for queries like
this all over the place :) plus... mysql 5.1 partitions will solve my problems
almost entirely
					res = qry("SELECT u.`uid`, `firstname`, `lastname`, `fullname`, `pic` FROM
`group_members` LEFT JOIN `users` u ON (u.`uid` = `id`) WHERE `type` = '" ~
s_gid_user_friends ~ "' AND `gid` = '^' AND `id` != '^'", *friends_gid, uid);
					count = res.length;
				}
				
				return generic_loop;
			});
	}
	
	
	uint* friends_gid;
	uint this_init(PNL* pnl, void*[char[]] args) {
		pnl.registerString(name ~ ".firstname", &firstname);
		pnl.registerString(name ~ ".lastname", &lastname);
		pnl.registerString(name ~ ".fullname", &fullname);
		pnl.registerString(name ~ ".pic", &pic);
		
		if("friends_gid" in args) {
			friends_gid = cast(uint*)args["friends_gid"];
		}
		
		return 0;
	}
	
	void this_set() {
		Base.zid = toUint(res[current][0]);
		firstname = res[current][1];
		lastname = res[current][2];
		fullname = res[current][3];
		pic = res[current][4];
	}
	
	mixin Generic!(this_init, this_set) G;
	alias G.loop generic_loop;
	alias G.pnl_init generic_init;
	alias G.current current;
	alias G.reload reload;
	alias G.total total;
}

The other thing that's written in D are the functions (things that modify the
database). Our database doesn't use cluster, it uses replication, so every
write has to go through the single master -- for this reason, all qry (full
table), q1r (one row), q1c (one column) and q1v (one value) talk to the
database at localhost, and q(for writing) talks to the master. q(...) can only
be found in functions.d ... which is good to prevent possible failures.

functions are simple. All data coming in comes to the POST/GET associative
array, and the function is called with the query or post string func=logout or
func=postwcomment

case "logout":
	if(uid) {
		q("UPDATE `session` SET `online` = 0 WHERE `uid` = ^ AND `sid` = ^", uid,
sid);
		q("UPDATE `users` SET `last_logout` = ^ WHERE `uid` = ^", request_time, uid);
		uid = zid = 0;
		is_me = false;
	}
break;
case "postwcomment":
	if(uid) {
		char[]* wcomment = ("wcomment" in POST);
		if(wcomment) {
			// There's no injection here, strings are automatically escaped with the
query functions... :)
			q("INSERT INTO `profilewall` ( `profile_uid` , `poster_uid` , `wid` ,
`text`, `time` )"
					"VALUES ('^', '^', '', '^', '^');", zid, uid, *wcomment, request_time);
		}
	}
break;



Still interested? Feedback? Thoughts?
Jan 22 2007
prev sibling parent Paul Findlay <r.lph50+d gmail.com> writes:
 if you guys are really interested, I'll write some samples up to get 
 feedback before opening it.
That sounds amazing!! Examples would be great, though. :)
Yeah, it does sound pretty interesting :) - Paul
Jan 22 2007
prev sibling parent reply =?UTF-8?B?SnVsaW8gQ8Opc2FyIENhcnJhc2NhbCBVcnF1aWpv?= writes:
Antti Holvikari wrote:
 On 1/22/07, Hasan Aljudy <hasan.aljudy gmail.com> wrote:
 I wish we'd have a framework similar to Django, however, come to think
 of it .. I'd rather use Django than wait for a D clone .. which I don't
 think can be more powerful since D doesn't have any dynamic reflection
 capabilities.
Hmm, hope I'm not missing something here but why would you need reflection for that?
Django uses python's reflections capabilities to bind classes to SQL tables and build administration interfaces in ways that I don't thing even the compile time reflection capabilities of D can implement. The following code taken from the Django tutorial admin interfaces for this Master/Details model with all the features you will expect: Pagination, searching, basic validation and form field grouping. Adding filtering and custom validation is really easy. from django.db import models class Poll(models.Model): question = models.CharField(maxlength=200) pub_date = models.DateTimeField('date published') class Admin: fields = ( (None, {'fields': ('question',)}), ('Date information', {'fields': ('pub_date',)}), ) class Choice(models.Model): poll = models.ForeignKey(Poll) choice = models.CharField(maxlength=200) votes = models.IntegerField() class Admin: pass As you can see, Django uses reflection to build all of this functionality with just some hints from the programmer.
Jan 22 2007
parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Julio César Carrascal Urquijo wrote:
 Antti Holvikari wrote:
 On 1/22/07, Hasan Aljudy <hasan.aljudy gmail.com> wrote:
 I wish we'd have a framework similar to Django, however, come to think
 of it .. I'd rather use Django than wait for a D clone .. which I don't
 think can be more powerful since D doesn't have any dynamic reflection
 capabilities.
Hmm, hope I'm not missing something here but why would you need reflection for that?
Django uses python's reflections capabilities to bind classes to SQL tables and build administration interfaces in ways that I don't thing even the compile time reflection capabilities of D can implement. The following code taken from the Django tutorial admin interfaces for this Master/Details model with all the features you will expect: Pagination, searching, basic validation and form field grouping. Adding filtering and custom validation is really easy. from django.db import models class Poll(models.Model): question = models.CharField(maxlength=200) pub_date = models.DateTimeField('date published') class Admin: fields = ( (None, {'fields': ('question',)}), ('Date information', {'fields': ('pub_date',)}), ) class Choice(models.Model): poll = models.ForeignKey(Poll) choice = models.CharField(maxlength=200) votes = models.IntegerField() class Admin: pass As you can see, Django uses reflection to build all of this functionality with just some hints from the programmer.
I'm wondering whether this can be achieved now with the new mixin semantics. The usage maybe ugly: mixin( models.Model!("poll", models.CharField!("question"), models.DateTimeField!("pubdate") ) ); But maybe this can generate some rich code .. I wish had the time and experience to take on this. Anyone willing to give it a try?
Feb 06 2007
parent Kirk McDonald <kirklin.mcdonald gmail.com> writes:
Hasan Aljudy wrote:
 
 
 Julio César Carrascal Urquijo wrote:
 Antti Holvikari wrote:
 On 1/22/07, Hasan Aljudy <hasan.aljudy gmail.com> wrote:
 I wish we'd have a framework similar to Django, however, come to think
 of it .. I'd rather use Django than wait for a D clone .. which I don't
 think can be more powerful since D doesn't have any dynamic reflection
 capabilities.
Hmm, hope I'm not missing something here but why would you need reflection for that?
Django uses python's reflections capabilities to bind classes to SQL tables and build administration interfaces in ways that I don't thing even the compile time reflection capabilities of D can implement. The following code taken from the Django tutorial admin interfaces for this Master/Details model with all the features you will expect: Pagination, searching, basic validation and form field grouping. Adding filtering and custom validation is really easy. from django.db import models class Poll(models.Model): question = models.CharField(maxlength=200) pub_date = models.DateTimeField('date published') class Admin: fields = ( (None, {'fields': ('question',)}), ('Date information', {'fields': ('pub_date',)}), ) class Choice(models.Model): poll = models.ForeignKey(Poll) choice = models.CharField(maxlength=200) votes = models.IntegerField() class Admin: pass As you can see, Django uses reflection to build all of this functionality with just some hints from the programmer.
I'm wondering whether this can be achieved now with the new mixin semantics. The usage maybe ugly: mixin( models.Model!("poll", models.CharField!("question"), models.DateTimeField!("pubdate") ) ); But maybe this can generate some rich code .. I wish had the time and experience to take on this. Anyone willing to give it a try?
I was playing with this a couple weeks ago, even without these new mixins. I started playing with a proof-of-concept, but got distracted by other things. I described some of my efforts in this forum thread: http://dsource.org/forums/viewtopic.php?t=2283 -- Kirk McDonald Pyd: Wrapping Python with D http://pyd.dsource.org
Feb 06 2007