www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [GSOC] Database API

reply Christian Manning <cmanning999 gmail.com> writes:
Hi,
I'm a second year student at De Montfort University studying Computer 
Science. I am very much interested in working on the database API idea 
that is proposed at 
http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DatabaseAPI (I was 
also quite interested in the containers idea, but it looks like someone 
else wants to do that)

I have limited experience with database libraries but have begun to 
study the JDBC API (in my small amount of spare time only as term 
doesn't finish for another 2 weeks for me and I still have assignments).

Has this idea/project been assigned a mentor? I'd like to ask them and 
the list, what's the best thing for me to do right now to prepare for this?

Thanks
Chris
Mar 24 2011
next sibling parent reply Trass3r <un known.com> writes:
 Has this idea/project been assigned a mentor? I'd like to ask them and  
 the list, what's the best thing for me to do right now to prepare for  
 this?



You could also have a look at http://dsource.org/projects/ddbi
This shows some past efforts to create database interfaces.
Mar 24 2011
next sibling parent reply Christian Manning <cmanning999 gmail.com> writes:
On 24/03/2011 17:58, Trass3r wrote:

 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for
 this?



 You could also have a look at http://dsource.org/projects/ddbi
 This shows some past efforts to create database interfaces.



This can be of some inspiration and also brings up a point about how 
this project would be executed, ie. is this to be part of phobos or a 
totally separate project? I'm guessing to be Digital Mars supported it's 
more likely wanted to be a part of phobos, but could someone clarify on 
this?

Another question about Digital Mars' GSOC involvement: when it's 
application time do I submit a proposal to someone directly, to this 
list, or to the GSOC site at http://www.google-melange.com ?

Thanks
Chris
Mar 25 2011
next sibling parent Christian Manning <cmanning999 gmail.com> writes:
On 25/03/2011 18:45, Jonathan M Davis wrote:

 On 24/03/2011 17:58, Trass3r wrote:

 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for
 this?



 You could also have a look at http://dsource.org/projects/ddbi
 This shows some past efforts to create database interfaces.



 This can be of some inspiration and also brings up a point about how
 this project would be executed, ie. is this to be part of phobos or a
 totally separate project? I'm guessing to be Digital Mars supported it's
 more likely wanted to be a part of phobos, but could someone clarify on
 this?



 What I would expect to be the case would be that it would be implemented with
 the idea that it would ideally be put into Phobos but that it wouldn't
 necessarily make it into Phobos. Any major new functionality that gets added
 to Phobos at this point needs to be reviewed by the folks on this newsgroup
 and get voted in. So, it's possible that a fairly solid solution could be
 created which would not be acceptable in Phobos for one reason or another, at
 which point it would have to either be fixed so that it would be acceptable in
 Phobos or be left as a 3rd party library.

 So, ideally stuff like a good database solution for D would make it into
 Phobos, but whether it actually makes it in depends a great deal on the actual
 solution.

 - Jonathan M Davis



Understood.

My last deadline is Monday, so after that I should have much more time 
to concentrate on this to plan out what I'd like to do and submit a 
proper proposal.

Chris
Mar 25 2011
prev sibling next sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/03/2011 00:04, Trass3r wrote:

 Another question about Digital Mars' GSOC involvement: when it's
 application time do I submit a proposal to someone directly, to this
 list, or to the GSOC site at http://www.google-melange.com ?



 You always have to apply via the google site.
 I'm sure I read that somewhere in the FAQs.



Yeah, exactly. Technically you don't strictly need to contact the 
organization before hand to submit the proposal, nor does the proposal 
have to be related to the ideas presented by the organization, the 
student can bring forth his own ideas.
But later the organization will rank each student proposal to eventually 
decide which ones get accepted, so of course it helps that the student 
has discussed the proposal before hand.

-- 
Bruno Medeiros - Software Engineer
Mar 26 2011
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 03/25/2011 09:02 AM, Christian Manning wrote:

 On 24/03/2011 17:58, Trass3r wrote:

 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for
 this?



 You could also have a look at http://dsource.org/projects/ddbi
 This shows some past efforts to create database interfaces.



 This can be of some inspiration and also brings up a point about how
 this project would be executed, ie. is this to be part of phobos or a
 totally separate project? I'm guessing to be Digital Mars supported it's
 more likely wanted to be a part of phobos, but could someone clarify on
 this?



Either way would be acceptable.


 Another question about Digital Mars' GSOC involvement: when it's
 application time do I submit a proposal to someone directly, to this
 list, or to the GSOC site at http://www.google-melange.com ?



You'll submit via Google's interface. See 
http://d-programming-language.org/gsoc2011.html for details. Submission 
opens tomorrow.


Andrei
Mar 27 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
 On 24/03/2011 17:58, Trass3r wrote:

 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for
 this?



 You could also have a look at http://dsource.org/projects/ddbi
 This shows some past efforts to create database interfaces.



 This can be of some inspiration and also brings up a point about how
 this project would be executed, ie. is this to be part of phobos or a
 totally separate project? I'm guessing to be Digital Mars supported it's
 more likely wanted to be a part of phobos, but could someone clarify on
 this?



What I would expect to be the case would be that it would be implemented with 
the idea that it would ideally be put into Phobos but that it wouldn't 
necessarily make it into Phobos. Any major new functionality that gets added 
to Phobos at this point needs to be reviewed by the folks on this newsgroup 
and get voted in. So, it's possible that a fairly solid solution could be 
created which would not be acceptable in Phobos for one reason or another, at 
which point it would have to either be fixed so that it would be acceptable in 
Phobos or be left as a 3rd party library.

So, ideally stuff like a good database solution for D would make it into 
Phobos, but whether it actually makes it in depends a great deal on the actual 
solution.

- Jonathan M Davis
Mar 25 2011
prev sibling parent Trass3r <un known.com> writes:
 Another question about Digital Mars' GSOC involvement: when it's  
 application time do I submit a proposal to someone directly, to this  
 list, or to the GSOC site at http://www.google-melange.com ?



You always have to apply via the google site.
I'm sure I read that somewhere in the FAQs.
Mar 25 2011
prev sibling next sibling parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Christian Manning wrote:

 Hi,
 I'm a second year student at De Montfort University studying Computer
 Science. I am very much interested in working on the database API idea
 that is proposed at
 http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DatabaseAPI (I was
 also quite interested in the containers idea, but it looks like someone
 else wants to do that)

 I have limited experience with database libraries but have begun to
 study the JDBC API (in my small amount of spare time only as term
 doesn't finish for another 2 weeks for me and I still have assignments).

 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for this?

 Thanks
 Chris



I'm at finishing stage of my PostgreSQL client, written entirely in D. I 
will post sources in few days.

Before adding database support to Phobos, there are few important issues 
which obviously need further discussion. One of this issues are handling 
NULL values - essential thing in relational databases.

Although, there were some efforts to introduce standard Nullable or 
Optional type, they were aborted.

Currently I'm using this simple definition:

struct Nullable(T)
{
     Algebraic!(T, void*) Nullable;
     alias Nullable this;

     bool hasValue()
     {
         return cast(bool)Nullable.peek!(T);
     }

     bool isNull()
     {
         return cast(bool)!Nullable.peek!(T);
     }

     string toString()
     {
         T* t = Nullable.peek!T;
         return t ? to!string(*t) : "null";
     }
}

It flawlessy cooperates with Variant, which - I think - should 
officially support nulls as well. I would propose that standard Nullable 
type, should be a subtype of Algebraic union as above.

During writing my PostgreSQL interface, I also had some problems with 
Variant and std.conv.to functions. Currently std.conv doesn't support 
Variants, f.i. when using to!Variant(5), there are more than one toImpl 
match.

In case of DB api, Variants are also essential, because field types 
returned by database server aren't known at compile time.

To sum up, Phobos need some additional changes for database support, 
mainly addition of Nullable.
Mar 25 2011
parent reply Christian Manning <cmanning999 gmail.com> writes:
On 25/03/2011 14:56, Piotr Szturmaj wrote:

 To sum up, Phobos need some additional changes for database support,
 mainly addition of Nullable.



For now, couldn't this just be included with the database module?
Mar 25 2011
parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Christian Manning wrote:

 On 25/03/2011 14:56, Piotr Szturmaj wrote:

 To sum up, Phobos need some additional changes for database support,
 mainly addition of Nullable.



 For now, couldn't this just be included with the database module?



Yes. I think, currently this is the only option.
Mar 26 2011
prev sibling next sibling parent reply "Masahiro Nakagawa" <repeatedly gmail.com> writes:
On Fri, 25 Mar 2011 02:48:14 +0900, Christian Manning  
<cmanning999 gmail.com> wrote:


 Hi,
 I'm a second year student at De Montfort University studying Computer  
 Science. I am very much interested in working on the database API idea  
 that is proposed at  
 http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DatabaseAPI (I was  
 also quite interested in the containers idea, but it looks like someone  
 else wants to do that)

 I have limited experience with database libraries but have begun to  
 study the JDBC API (in my small amount of spare time only as term  
 doesn't finish for another 2 weeks for me and I still have assignments).

 Has this idea/project been assigned a mentor? I'd like to ask them and  
 the list, what's the best thing for me to do right now to prepare for  
 this?



Currently, many databases exist.

* SQL based: MySQL, PostgresSQL, SQLite, etc..
* KVS: Cassandra, HBase, Kumofs, Redis, etc...
* Document Oriented: MongoDB, CouchDB, etc...

I think next Database API should support these types.
AFAIK, DDBI supports only SQL based interface.

In Ruby, Rails 3 has Arel.
Arel seems to be a good query abstraction.

https://github.com/rails/arel

Arel's concept may help you :)


Masahiro
Mar 25 2011
next sibling parent Christian Manning <cmanning999 gmail.com> writes:
On 25/03/2011 23:44, Masahiro Nakagawa wrote:

 In Ruby, Rails 3 has Arel.
 Arel seems to be a good query abstraction.
 
 https://github.com/rails/arel
 
 Arel's concept may help you :)
 
 
 Masahiro



Now that's an interesting way of going about things. Although, its
description suggests that it's a framework for building frameworks which
isn't really what I'm after as the database API should be able to be
used by anybody easily. I don't know Ruby at all but in some of those
examples, the syntax is quite neat.

Chris
Mar 26 2011
prev sibling parent reply David Nadlinger <see klickverbot.at> writes:
On 3/26/11 12:44 AM, Masahiro Nakagawa wrote:

 Currently, many databases exist.

 * SQL based: MySQL, PostgresSQL, SQLite, etc..
 * KVS: Cassandra, HBase, Kumofs, Redis, etc...
 * Document Oriented: MongoDB, CouchDB, etc...

 I think next Database API should support these types.



I don't think this is the way to go, as there is a huge conceptional 
difference between relational database systems and »NoSQL« systems such 
as, say, CouchDB or Cassandra.

How would such a unified API look like?

David
Mar 26 2011
parent Daniel Gibson <metalcaedes gmail.com> writes:
Am 29.03.2011 01:18, schrieb Masahiro Nakagawa:

 On Sun, 27 Mar 2011 07:11:01 +0900, David Nadlinger <see klickverbot.at>
 wrote:


 On 3/26/11 12:44 AM, Masahiro Nakagawa wrote:

 Currently, many databases exist.

 * SQL based: MySQL, PostgresSQL, SQLite, etc..
 * KVS: Cassandra, HBase, Kumofs, Redis, etc...
 * Document Oriented: MongoDB, CouchDB, etc...

 I think next Database API should support these types.



 I don't think this is the way to go, as there is a huge conceptional
 difference between relational database systems and »NoSQL« systems
 such as, say, CouchDB or Cassandra.



 It's an open question.

 Ruby / Rails has Arel (note that Arel is data access abstraction,
 not database access abstraction. But Database access is a part of data
 access).
 Haskell has Database.Persistent.
 I think other languages will support similar library in the future.



I think proper support for relational databases with SQL is more 
important right now.
Once it is done some abstraction for generic data access can be 
developed, using the relational database support for these databases and 
something else (probably more specific to each NoSQL-Database, because 
AFAIK they are even less compatible to each other than relational DBs) 
for NoSQL-DBs.

Cheers,
- Daniel
Mar 29 2011
prev sibling parent "Masahiro Nakagawa" <repeatedly gmail.com> writes:
On Sun, 27 Mar 2011 07:11:01 +0900, David Nadlinger <see klickverbot.at>=
  =

wrote:


 On 3/26/11 12:44 AM, Masahiro Nakagawa wrote:

 Currently, many databases exist.

 * SQL based: MySQL, PostgresSQL, SQLite, etc..
 * KVS: Cassandra, HBase, Kumofs, Redis, etc...
 * Document Oriented: MongoDB, CouchDB, etc...

 I think next Database API should support these types.



 I don't think this is the way to go, as there is a huge conceptional  =




 difference between relational database systems and =C2=BBNoSQL=C2=AB s=





 as, say, CouchDB or Cassandra.



It's an open question.

Ruby / Rails has Arel (note that Arel is data access abstraction,
not database access abstraction. But Database access is a part of data  =

access).
Haskell has Database.Persistent.
I think other languages will support similar library in the future.


 How would such a unified API look like?



I don't have a detail.
I already implemented it if I had a detailed design.
In my D tasks, database related task is low priority...
Mar 28 2011