www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Domain Specific Languages in D; was: C++, D: Dinosaurs?

reply Aarti_pl <aarti interia.pl> writes:
Nick Sabalausky pisze:
 "Piotrek" <starpit tlen.pl> wrote in message
 news:geo14c$ol9$1 digitalmars.com...
 Nick Sabalausky wrote:
 [...]The concept of domain-specific languages is ultra-trendy these 



 (probably promoted by the same knuckleheads that hailed things like
 pure-OO, pure-functional, and Extreme Programming as silver 



 I consider domain-specific languages to be purely a symptom of a 



 need for a better general purpose language.

with pascal then moved to c/c++ at university but I abandoned it


 of its ugliness and non intuitive application development. Then I found
 working with PHP/JAVA much easier but I was disappointed by its
 fundamentals (please let me forget about PHP as a programming 


 And when I was left without any hope I saw a Bright light. :D


(....)
 I agree. I'd much rather just use a sensible API (DBMS-agnostic of 

 Pasting together SQL snippets is just plain ugly, and why dynamically
 generate code just for the DBMS to go and parse it? I mean, geez, 

 the middleman and pass the request directly. LINQ is better, but I'm 

 not convinced it's the right approach. TSQL stored procedures are 

 I'd rather, well, just write in a more "normal" language.  And then, of
 course, there's the bizarre "insert" vs. "update" syntax 

 general incompatibilities across vendors (ex: insert a row and 

 auto-gen id <- which is really a rather common thing to need to do,
 particularly if you've got any foreign keys in the DB design, which 

 rather common).

 To be fair though, I have to say that without some of the advanced 

 we've been starting to see in languages like D, C#, Python and Ruby, 

 an API for any non-trivial queries could potentially get ugly.

Above subject is very interesting. I didn't know for example about LINQ, but have very similar idea to implement Sql queries as specific objects, which are later executed. Currently I implement something similar what is already working, and I have to say, it is working very good. My implementation right now is in Java (because of easy way to make refactorings with Eclipse), but I want to port it into D, and put into my libraries at DSource (Doost project). For example LINQ statement: ------------------------------ var query1 = from p in people where p.Age > 20 orderby p.Age descending; ------------------------------ will look in my implementation like below: ------------------------------ auto query = Select(people) .Where(EqualsOrMore(people.age, 20)) .OrderBy(people.age, DESC) .OrderBy(people.name, ASC); ------------------------------ So it's quite similar to SQL standard, but without drawbacks of building SQL's as strings. ------------------------------ As I said before, my implementation is already working. But still I see many problems in OO languages (Java, C++, D) which disallows to define such sublanguage in a clear way. Let me enumerate a few: 1. Lack of opImplicitCast() I see this part as very important and blocking nicer solutions in many areas. Hopefully D will get it implemented sooner than later. For example Where clause in SQL is defined as following: Where(expression) expression can be e.g. equality, column or select statement, so there is need to put all this cases when defining method "Where". With opImplicitCastTo() it would be possible to define it in e.g. Column. Then Where will take only objects of Expression. So we get better encapsulation. opImplicitCast would also solve problems with slicing utf8 char arrays, as it would be possible to write class/struct with opImplicitCast which will convert to char arrays, keeping high level slicing in class/struct. 2. Lack of finer control over operators overloading Current D implementation makes it impossible to use operators in SQL OO sublanguage. Although there is opEquals and opCmp you can not really use them as using them you can not distinguish what exactly operation was requested and you can not return different type than predefined e.g. bool in case of opEquals. Above have of course its merits, but also disallows to define nice looking SQL queries. To solve this problem I would suggest to allow definition of additional operators like below: opRawEquals (opRawNotEquals ??) opRawMore opRawLess etc. then only one: opEquals or opRawEquals can be defined. It would be also nice to have possibility to define user operators, as it seems that it is natural for humans to use some operators in their infix form. 3. Impossibility to statically control meta language contructs. Sql has its own syntax rules and currently they can not be enforced on compile time. E.g. in select statement you would expect to have OrderBy after Where, not other way. Currently I can enforce order of clauses only on runtime, although it should be probably doable to enforce it on compile time. Currently it is also not possible to enforce that e.g. some method must be called: Query query = Select(id).From(table); Imagine that you want to enforce that in above query From() must be always called (Select is an object, From is method on this object). So what's more is lacking in D, to absorb even more DSL languages? Thoughts? PS. If someone is interested in above subject and wants to help in porting project from Java and improving it - please contact me on my e-mail address. Regards Marcin Kuszczak (aarti_pl)
Nov 04 2008
next sibling parent reply =?iso-8859-1?Q?Julio=20C=e9sar=20Carrascal=20Urquijo?= <jcarrascal gmail.com> writes:
Hello Aarti_pl,

 3. Impossibility to statically control meta language contructs.
 
 Sql has its own syntax rules and currently they can not be enforced on
 compile time.
 
 E.g. in select statement you would expect to have OrderBy after Where,
 not other way.

Instead of returning the full query object you could return a "Fragment" query. Something like this: class WhereFragment { this(Query q) {} WhereFragment And(Expression e) {} OrderByFragment OrderBy(Column[] columns) {} }
Currently I can enforce order of clauses only on
 runtime, although it should be probably doable to enforce it on
 compile time. Currently it is also not possible to enforce that e.g.
 some method must be called:

 Query query = Select(id).From(table);
 
 Imagine that you want to enforce that in above query From() must be
 always called (Select is an object, From is method on this object).

You should probably do as LINQ to SQL and start the query with the FROM statement instead. That way you can cascade the type and help auto-complete in IDEs. But yes, I don't think it's currently possible to enforce a method to be called.
Nov 04 2008
parent reply Aarti_pl <aarti interia.pl> writes:
Julio CÚsar Carrascal Urquijo pisze:
 Hello Aarti_pl,
 
 3. Impossibility to statically control meta language contructs.

 Sql has its own syntax rules and currently they can not be enforced on
 compile time.

 E.g. in select statement you would expect to have OrderBy after Where,
 not other way.

Instead of returning the full query object you could return a "Fragment" query. Something like this: class WhereFragment { this(Query q) {} WhereFragment And(Expression e) {} OrderByFragment OrderBy(Column[] columns) {} }

Yes, something like this would be possible. But it would start to be very complicated, unmanageable and not so useful soon. Implementing it in such a way for every DSL will be very hard. Also there are cases in normal programs where it would be useful to have such checks. Currently I use simple runtime checks (somewhat simplified, pseudocodish and not tested code below): class SelectStatement { string[] next; this() { next = ["Where", "From"]; //All possible options after construction } public SelectStatement Where(SqlExpression exp) { if ("Where" !in next) throw Exception("Syntax error"); ... next = ["OrderBy"]; } } Probably it should be possible to implement something similar in compiler to make checks on compile time. If some variable has assigned object of such a class compiler should trace this variable and check if methods are called properly (in order, that they *are* called etc.) I don't know how easy / difficult is to implement something like this in compiler, but I think it may be difficult.
 
 Currently I can enforce order of clauses only on
 runtime, although it should be probably doable to enforce it on
 compile time. Currently it is also not possible to enforce that e.g.
 some method must be called:

 Query query = Select(id).From(table);

 Imagine that you want to enforce that in above query From() must be
 always called (Select is an object, From is method on this object).

You should probably do as LINQ to SQL and start the query with the FROM statement instead. That way you can cascade the type and help auto-complete in IDEs.

Well, I don't see problem here with my approach... SQL has a lot of options after SELECT keyword and before WHERE. How can I use them using only From?
 But yes, I don't think it's currently possible to 
 enforce a method to be called.

BR Marcin Kuszczak (aarti_pl)
Nov 04 2008
parent reply =?iso-8859-1?Q?Julio=20C=e9sar=20Carrascal=20Urquijo?= <jcarrascal gmail.com> writes:
Hello Aarti_pl,

 Yes, something like this would be possible. But it would start to be
 very complicated, unmanageable and not so useful soon. Implementing it
 in such a way for every DSL will be very hard.

Fragment objects are used in SubSonic so I think it is manageable at least for the SQL case. One thing I should probably have shown in the example is that Fragment objects just route calls to the underlying Query object so they are small and don't have any logic: class WhereFragment { this(Query query) { m_query = query; } OrderByFragment OrderBy(Column[] columns) { m_query.OrderBy(columns); } Query m_query; } In your case it would be a template of the Table class, of course. Also, you would need create a small number of them (WhereFragment, OrderByFragment, SelectFragment).
 Also there are cases in
 normal programs where it would be useful to have such checks.

Agree. For example, GUI controls sometimes requires you to set several properties to specific values to get a specific effect. This could benefit from having this kind of support from the language. Though I don't think this is a pressing issue.
 Currently I use simple runtime checks (somewhat simplified,
 pseudocodish and not tested code below):
 
 class SelectStatement {
 string[] next;
 this() {
 next = ["Where", "From"]; //All possible options after
 construction
 }
 public SelectStatement Where(SqlExpression exp) {
 if ("Where" !in next) throw Exception("Syntax error");
 ...
 
 next = ["OrderBy"];
 }
 }
 Probably it should be possible to implement something similar in
 compiler to make checks on compile time. If some variable has assigned
 object of such a class compiler should trace this variable and check
 if methods are called properly (in order, that they *are* called etc.)

I would say that fragment objects are simpler than littering your methods with these checks.
 You should probably do as LINQ to SQL and start the query with the
 FROM statement instead. That way you can cascade the type and help
 auto-complete in IDEs.
 

options after SELECT keyword and before WHERE. How can I use them using only From?

I meant something like this: auto q = From(Employee) .Where(Employee.Email, Is.EqualTo("jcarrascal gm...")) .Select(Employee.ID); This way the Employee type in the From() would cascade to the Where() and the Select() and you could add compile time checks to allow only columns of that table to be used in there. One question: Do you library supports (inner/outer) Joins or have plans to support them? Thanks
Nov 04 2008
parent Marcin Kuszczak <aarti_please_no spam_interia.pl> writes:
Julio C├ęsar Carrascal Urquijo wrote:

 Hello Aarti_pl,
 
 Yes, something like this would be possible. But it would start to be
 very complicated, unmanageable and not so useful soon. Implementing it
 in such a way for every DSL will be very hard.

Fragment objects are used in SubSonic so I think it is manageable at least for the SQL case.

I didn't know about SubSonic. I was thinking that only project which tries to use OO SQL queries was Ultimate++. So, thanks for reference!
 One thing I should probably have shown in the example is that Fragment
 objects just route calls to the underlying Query object so they are small
 and don't have any logic:
 
 class WhereFragment
 {
     this(Query query) { m_query = query; }
 
     OrderByFragment OrderBy(Column[] columns)
     { 
         m_query.OrderBy(columns);
     }
 
     Query m_query;
 }
 
 In your case it would be a template of the Table class, of course. Also,
 you would need create a small number of them (WhereFragment,
 OrderByFragment, SelectFragment).

This is interesting technique. But also IMHO not always usable as you can not finish statement with WhereFragment when your result is SelectStatement. Wait, I just realized that opImplicitCast will be also very helpful in this case as WhereFragment can be implicitly casted to Query :-D
 Also there are cases in
 normal programs where it would be useful to have such checks.

Agree. For example, GUI controls sometimes requires you to set several properties to specific values to get a specific effect. This could benefit from having this kind of support from the language. Though I don't think this is a pressing issue.
 Currently I use simple runtime checks (somewhat simplified,
 pseudocodish and not tested code below):
 
 class SelectStatement {
 string[] next;
 this() {
 next = ["Where", "From"]; //All possible options after
 construction
 }
 public SelectStatement Where(SqlExpression exp) {
 if ("Where" !in next) throw Exception("Syntax error");
 ...
 
 next = ["OrderBy"];
 }
 }
 Probably it should be possible to implement something similar in
 compiler to make checks on compile time. If some variable has assigned
 object of such a class compiler should trace this variable and check
 if methods are called properly (in order, that they *are* called etc.)

I would say that fragment objects are simpler than littering your methods with these checks.

You are probably right here. But I don't have in Java and still in D opImplicitCast to the rescue and would have to add to fragments artificial e.g. .apply() method which would always return query. My design is a bit different from SubSonic one. My queries are quite independent, and they do not evaluate automatically to results. They can instead be passed and modified in different places in program. When query is ready it can be changed into SQL by one of generators. Generator can create SQL for specific database. But it can also create hash to help caching queries. After that query is executed by one of available Executor. So in my design artificial .apply() will not be very nice addition....
 You should probably do as LINQ to SQL and start the query with the
 FROM statement instead. That way you can cascade the type and help
 auto-complete in IDEs.
 

options after SELECT keyword and before WHERE. How can I use them using only From?

I meant something like this: auto q = From(Employee) .Where(Employee.Email, Is.EqualTo("jcarrascal gm...")) .Select(Employee.ID); This way the Employee type in the From() would cascade to the Where() and the Select() and you could add compile time checks to allow only columns of that table to be used in there.

I see. I took a bit different way in this case. In my design From is optional as all tables are inferred from Select() (every column have defined its table, so it it easy).
 One question: Do you library supports (inner/outer) Joins or have plans to
 support them?

Yes. I have added them lately.
 Thanks

-- Regards Marcin Kuszczak (Aarti_pl) ------------------------------------- Ask me why I believe in Jesus - http://www.zapytajmnie.com (en/pl) Doost (port of few Boost libraries) - http://www.dsource.org/projects/doost/ -------------------------------------
Nov 04 2008
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Aarti_pl" <aarti interia.pl> wrote in message 
news:gepf4o$1118$1 digitalmars.com...
 Nick Sabalausky pisze:
 "Piotrek" <starpit tlen.pl> wrote in message
 news:geo14c$ol9$1 digitalmars.com...
 Nick Sabalausky wrote:
 [...]The concept of domain-specific languages is ultra-trendy these



 (probably promoted by the same knuckleheads that hailed things like
 pure-OO, pure-functional, and Extreme Programming as silver



 I consider domain-specific languages to be purely a symptom of a



 need for a better general purpose language.

with pascal then moved to c/c++ at university but I abandoned it


 of its ugliness and non intuitive application development. Then I found
 working with PHP/JAVA much easier but I was disappointed by its
 fundamentals (please let me forget about PHP as a programming


 And when I was left without any hope I saw a Bright light. :D


(....)
 I agree. I'd much rather just use a sensible API (DBMS-agnostic of

 Pasting together SQL snippets is just plain ugly, and why dynamically
 generate code just for the DBMS to go and parse it? I mean, geez,

 the middleman and pass the request directly. LINQ is better, but I'm

 not convinced it's the right approach. TSQL stored procedures are

 I'd rather, well, just write in a more "normal" language.  And then, of
 course, there's the bizarre "insert" vs. "update" syntax

 general incompatibilities across vendors (ex: insert a row and

 auto-gen id <- which is really a rather common thing to need to do,
 particularly if you've got any foreign keys in the DB design, which

 rather common).

 To be fair though, I have to say that without some of the advanced

 we've been starting to see in languages like D, C#, Python and Ruby,

 an API for any non-trivial queries could potentially get ugly.

Above subject is very interesting. I didn't know for example about LINQ, but have very similar idea to implement Sql queries as specific objects, which are later executed. Currently I implement something similar what is already working, and I have to say, it is working very good. My implementation right now is in Java (because of easy way to make refactorings with Eclipse), but I want to port it into D, and put into my libraries at DSource (Doost project). For example LINQ statement: ------------------------------ var query1 = from p in people where p.Age > 20 orderby p.Age descending; ------------------------------ will look in my implementation like below: ------------------------------ auto query = Select(people) .Where(EqualsOrMore(people.age, 20)) .OrderBy(people.age, DESC) .OrderBy(people.name, ASC); ------------------------------ So it's quite similar to SQL standard, but without drawbacks of building SQL's as strings. ------------------------------ As I said before, my implementation is already working. But still I see many problems in OO languages (Java, C++, D) which disallows to define such sublanguage in a clear way. Let me enumerate a few: 1. Lack of opImplicitCast() I see this part as very important and blocking nicer solutions in many areas. Hopefully D will get it implemented sooner than later. For example Where clause in SQL is defined as following: Where(expression) expression can be e.g. equality, column or select statement, so there is need to put all this cases when defining method "Where". With opImplicitCastTo() it would be possible to define it in e.g. Column. Then Where will take only objects of Expression. So we get better encapsulation. opImplicitCast would also solve problems with slicing utf8 char arrays, as it would be possible to write class/struct with opImplicitCast which will convert to char arrays, keeping high level slicing in class/struct. 2. Lack of finer control over operators overloading Current D implementation makes it impossible to use operators in SQL OO sublanguage. Although there is opEquals and opCmp you can not really use them as using them you can not distinguish what exactly operation was requested and you can not return different type than predefined e.g. bool in case of opEquals. Above have of course its merits, but also disallows to define nice looking SQL queries. To solve this problem I would suggest to allow definition of additional operators like below: opRawEquals (opRawNotEquals ??) opRawMore opRawLess etc. then only one: opEquals or opRawEquals can be defined. It would be also nice to have possibility to define user operators, as it seems that it is natural for humans to use some operators in their infix form. 3. Impossibility to statically control meta language contructs. Sql has its own syntax rules and currently they can not be enforced on compile time. E.g. in select statement you would expect to have OrderBy after Where, not other way. Currently I can enforce order of clauses only on runtime, although it should be probably doable to enforce it on compile time. Currently it is also not possible to enforce that e.g. some method must be called: Query query = Select(id).From(table); Imagine that you want to enforce that in above query From() must be always called (Select is an object, From is method on this object). So what's more is lacking in D, to absorb even more DSL languages? Thoughts? PS. If someone is interested in above subject and wants to help in porting project from Java and improving it - please contact me on my e-mail address. Regards Marcin Kuszczak (aarti_pl)

If we had extension methods, you could change: .Where(EqualsOrMore(people.age, 20)) to: .Where(people.age.EqualsOrMore(20)) Which is a little bit nicer. But I agree, a way to return a non-standard type from an operator overload would be even better. Something about both of those above reminds me of a unittesting tool I was looking at a while ago...can't remember what it was though...I *think* it was a unittesting tool...
Nov 04 2008
next sibling parent Aarti_pl <aarti interia.pl> writes:
Nick Sabalausky pisze:
 But I agree, a way to return a non-standard 
 type from an operator overload would be even better.

Unfortunately it would not be that easy in D. With current operator overloading method you can not distinguish between equals and not equals, as method equals makes comparison itself - it's not just notification for user program that some specific operator should be called. As I said before - current behavior makes sense, but it's also problem when you want to use &&, ||, == operators in SQL expressions (it definitely makes sense in such context and doesn't make user code look bad, but is not possible with schema choosen in D). It would be solved with below: 1. Allow additional operators in D, such as: T opSymbolEquals(U v1, V v2); //alt. T op==(U v1, V v2); T opSymbolNotEquals(U v1, V v2); //alt. T op!=(U v1, V v2); ... for cases where raw operators should be used. Then it will be possible to use only one standard opEquals() or opSymbolEquals in one class. 2. Allow defining user infix operators. Then it would be possible to define e.g. opAND() which is quite good for SQL queries. BR Marcin Kuszczak (aarti_pl)
Nov 05 2008
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 05 Nov 2008 00:02:36 +0300, Nick Sabalausky <a a.a> wrote:

 Something about both of those above reminds me of a unittesting tool I  
 was
 looking at a while ago...can't remember what it was though...I *think* it
 was a unittesting tool...

NUnit? Assert.That( myString, Is.EqualTo("Hello") );
Nov 05 2008
parent "Nick Sabalausky" <a a.a> writes:
"Denis Koroskin" <2korden gmail.com> wrote in message 
news:op.uj48si0oo7cclz proton.creatstudio.intranet...
 On Wed, 05 Nov 2008 00:02:36 +0300, Nick Sabalausky <a a.a> wrote:

 Something about both of those above reminds me of a unittesting tool I 
 was
 looking at a while ago...can't remember what it was though...I *think* it
 was a unittesting tool...

NUnit? Assert.That( myString, Is.EqualTo("Hello") );

Yea, I think that was it.
Nov 05 2008