www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [GSOC] Database API

reply Christian Manning <cmanning999 gmail.com> writes:
Hi,
I'm a second year student at De Montfort University studying Computer 
Science. I am very much interested in working on the database API idea 
that is proposed at 
http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DatabaseAPI (I was 
also quite interested in the containers idea, but it looks like someone 
else wants to do that)

I have limited experience with database libraries but have begun to 
study the JDBC API (in my small amount of spare time only as term 
doesn't finish for another 2 weeks for me and I still have assignments).

Has this idea/project been assigned a mentor? I'd like to ask them and 
the list, what's the best thing for me to do right now to prepare for this?

Thanks
Chris
Mar 24 2011
next sibling parent reply Trass3r <un known.com> writes:
 Has this idea/project been assigned a mentor? I'd like to ask them and  
 the list, what's the best thing for me to do right now to prepare for  
 this?

You could also have a look at http://dsource.org/projects/ddbi This shows some past efforts to create database interfaces.
Mar 24 2011
next sibling parent reply Christian Manning <cmanning999 gmail.com> writes:
On 24/03/2011 17:58, Trass3r wrote:
 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for
 this?

You could also have a look at http://dsource.org/projects/ddbi This shows some past efforts to create database interfaces.

This can be of some inspiration and also brings up a point about how this project would be executed, ie. is this to be part of phobos or a totally separate project? I'm guessing to be Digital Mars supported it's more likely wanted to be a part of phobos, but could someone clarify on this? Another question about Digital Mars' GSOC involvement: when it's application time do I submit a proposal to someone directly, to this list, or to the GSOC site at http://www.google-melange.com ? Thanks Chris
Mar 25 2011
next sibling parent Christian Manning <cmanning999 gmail.com> writes:
On 25/03/2011 18:45, Jonathan M Davis wrote:
 On 24/03/2011 17:58, Trass3r wrote:
 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for
 this?

You could also have a look at http://dsource.org/projects/ddbi This shows some past efforts to create database interfaces.

This can be of some inspiration and also brings up a point about how this project would be executed, ie. is this to be part of phobos or a totally separate project? I'm guessing to be Digital Mars supported it's more likely wanted to be a part of phobos, but could someone clarify on this?

What I would expect to be the case would be that it would be implemented with the idea that it would ideally be put into Phobos but that it wouldn't necessarily make it into Phobos. Any major new functionality that gets added to Phobos at this point needs to be reviewed by the folks on this newsgroup and get voted in. So, it's possible that a fairly solid solution could be created which would not be acceptable in Phobos for one reason or another, at which point it would have to either be fixed so that it would be acceptable in Phobos or be left as a 3rd party library. So, ideally stuff like a good database solution for D would make it into Phobos, but whether it actually makes it in depends a great deal on the actual solution. - Jonathan M Davis

Understood. My last deadline is Monday, so after that I should have much more time to concentrate on this to plan out what I'd like to do and submit a proper proposal. Chris
Mar 25 2011
prev sibling next sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/03/2011 00:04, Trass3r wrote:
 Another question about Digital Mars' GSOC involvement: when it's
 application time do I submit a proposal to someone directly, to this
 list, or to the GSOC site at http://www.google-melange.com ?

You always have to apply via the google site. I'm sure I read that somewhere in the FAQs.

Yeah, exactly. Technically you don't strictly need to contact the organization before hand to submit the proposal, nor does the proposal have to be related to the ideas presented by the organization, the student can bring forth his own ideas. But later the organization will rank each student proposal to eventually decide which ones get accepted, so of course it helps that the student has discussed the proposal before hand. -- Bruno Medeiros - Software Engineer
Mar 26 2011
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 03/25/2011 09:02 AM, Christian Manning wrote:
 On 24/03/2011 17:58, Trass3r wrote:
 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for
 this?

You could also have a look at http://dsource.org/projects/ddbi This shows some past efforts to create database interfaces.

This can be of some inspiration and also brings up a point about how this project would be executed, ie. is this to be part of phobos or a totally separate project? I'm guessing to be Digital Mars supported it's more likely wanted to be a part of phobos, but could someone clarify on this?

Either way would be acceptable.
 Another question about Digital Mars' GSOC involvement: when it's
 application time do I submit a proposal to someone directly, to this
 list, or to the GSOC site at http://www.google-melange.com ?

You'll submit via Google's interface. See http://d-programming-language.org/gsoc2011.html for details. Submission opens tomorrow. Andrei
Mar 27 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
 On 24/03/2011 17:58, Trass3r wrote:
 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for
 this?

You could also have a look at http://dsource.org/projects/ddbi This shows some past efforts to create database interfaces.

This can be of some inspiration and also brings up a point about how this project would be executed, ie. is this to be part of phobos or a totally separate project? I'm guessing to be Digital Mars supported it's more likely wanted to be a part of phobos, but could someone clarify on this?

What I would expect to be the case would be that it would be implemented with the idea that it would ideally be put into Phobos but that it wouldn't necessarily make it into Phobos. Any major new functionality that gets added to Phobos at this point needs to be reviewed by the folks on this newsgroup and get voted in. So, it's possible that a fairly solid solution could be created which would not be acceptable in Phobos for one reason or another, at which point it would have to either be fixed so that it would be acceptable in Phobos or be left as a 3rd party library. So, ideally stuff like a good database solution for D would make it into Phobos, but whether it actually makes it in depends a great deal on the actual solution. - Jonathan M Davis
Mar 25 2011
prev sibling parent Trass3r <un known.com> writes:
 Another question about Digital Mars' GSOC involvement: when it's  
 application time do I submit a proposal to someone directly, to this  
 list, or to the GSOC site at http://www.google-melange.com ?

You always have to apply via the google site. I'm sure I read that somewhere in the FAQs.
Mar 25 2011
prev sibling next sibling parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Christian Manning wrote:
 Hi,
 I'm a second year student at De Montfort University studying Computer
 Science. I am very much interested in working on the database API idea
 that is proposed at
 http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DatabaseAPI (I was
 also quite interested in the containers idea, but it looks like someone
 else wants to do that)

 I have limited experience with database libraries but have begun to
 study the JDBC API (in my small amount of spare time only as term
 doesn't finish for another 2 weeks for me and I still have assignments).

 Has this idea/project been assigned a mentor? I'd like to ask them and
 the list, what's the best thing for me to do right now to prepare for this?

 Thanks
 Chris

I'm at finishing stage of my PostgreSQL client, written entirely in D. I will post sources in few days. Before adding database support to Phobos, there are few important issues which obviously need further discussion. One of this issues are handling NULL values - essential thing in relational databases. Although, there were some efforts to introduce standard Nullable or Optional type, they were aborted. Currently I'm using this simple definition: struct Nullable(T) { Algebraic!(T, void*) Nullable; alias Nullable this; bool hasValue() { return cast(bool)Nullable.peek!(T); } bool isNull() { return cast(bool)!Nullable.peek!(T); } string toString() { T* t = Nullable.peek!T; return t ? to!string(*t) : "null"; } } It flawlessy cooperates with Variant, which - I think - should officially support nulls as well. I would propose that standard Nullable type, should be a subtype of Algebraic union as above. During writing my PostgreSQL interface, I also had some problems with Variant and std.conv.to functions. Currently std.conv doesn't support Variants, f.i. when using to!Variant(5), there are more than one toImpl match. In case of DB api, Variants are also essential, because field types returned by database server aren't known at compile time. To sum up, Phobos need some additional changes for database support, mainly addition of Nullable.
Mar 25 2011
parent reply Christian Manning <cmanning999 gmail.com> writes:
On 25/03/2011 14:56, Piotr Szturmaj wrote:
 To sum up, Phobos need some additional changes for database support,
 mainly addition of Nullable.

For now, couldn't this just be included with the database module?
Mar 25 2011
parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Christian Manning wrote:
 On 25/03/2011 14:56, Piotr Szturmaj wrote:
 To sum up, Phobos need some additional changes for database support,
 mainly addition of Nullable.

For now, couldn't this just be included with the database module?

Yes. I think, currently this is the only option.
Mar 26 2011
prev sibling next sibling parent reply "Masahiro Nakagawa" <repeatedly gmail.com> writes:
On Fri, 25 Mar 2011 02:48:14 +0900, Christian Manning  
<cmanning999 gmail.com> wrote:

 Hi,
 I'm a second year student at De Montfort University studying Computer  
 Science. I am very much interested in working on the database API idea  
 that is proposed at  
 http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DatabaseAPI (I was  
 also quite interested in the containers idea, but it looks like someone  
 else wants to do that)

 I have limited experience with database libraries but have begun to  
 study the JDBC API (in my small amount of spare time only as term  
 doesn't finish for another 2 weeks for me and I still have assignments).

 Has this idea/project been assigned a mentor? I'd like to ask them and  
 the list, what's the best thing for me to do right now to prepare for  
 this?

Currently, many databases exist. * SQL based: MySQL, PostgresSQL, SQLite, etc.. * KVS: Cassandra, HBase, Kumofs, Redis, etc... * Document Oriented: MongoDB, CouchDB, etc... I think next Database API should support these types. AFAIK, DDBI supports only SQL based interface. In Ruby, Rails 3 has Arel. Arel seems to be a good query abstraction. https://github.com/rails/arel Arel's concept may help you :) Masahiro
Mar 25 2011
next sibling parent Christian Manning <cmanning999 gmail.com> writes:
On 25/03/2011 23:44, Masahiro Nakagawa wrote:
 In Ruby, Rails 3 has Arel.
 Arel seems to be a good query abstraction.
 
 https://github.com/rails/arel
 
 Arel's concept may help you :)
 
 
 Masahiro

Now that's an interesting way of going about things. Although, its description suggests that it's a framework for building frameworks which isn't really what I'm after as the database API should be able to be used by anybody easily. I don't know Ruby at all but in some of those examples, the syntax is quite neat. Chris
Mar 26 2011
prev sibling parent reply David Nadlinger <see klickverbot.at> writes:
On 3/26/11 12:44 AM, Masahiro Nakagawa wrote:
 Currently, many databases exist.

 * SQL based: MySQL, PostgresSQL, SQLite, etc..
 * KVS: Cassandra, HBase, Kumofs, Redis, etc...
 * Document Oriented: MongoDB, CouchDB, etc...

 I think next Database API should support these types.

I don't think this is the way to go, as there is a huge conceptional difference between relational database systems and »NoSQL« systems such as, say, CouchDB or Cassandra. How would such a unified API look like? David
Mar 26 2011
parent Daniel Gibson <metalcaedes gmail.com> writes:
Am 29.03.2011 01:18, schrieb Masahiro Nakagawa:
 On Sun, 27 Mar 2011 07:11:01 +0900, David Nadlinger <see klickverbot.at>
 wrote:

 On 3/26/11 12:44 AM, Masahiro Nakagawa wrote:
 Currently, many databases exist.

 * SQL based: MySQL, PostgresSQL, SQLite, etc..
 * KVS: Cassandra, HBase, Kumofs, Redis, etc...
 * Document Oriented: MongoDB, CouchDB, etc...

 I think next Database API should support these types.

I don't think this is the way to go, as there is a huge conceptional difference between relational database systems and »NoSQL« systems such as, say, CouchDB or Cassandra.

It's an open question. Ruby / Rails has Arel (note that Arel is data access abstraction, not database access abstraction. But Database access is a part of data access). Haskell has Database.Persistent. I think other languages will support similar library in the future.

I think proper support for relational databases with SQL is more important right now. Once it is done some abstraction for generic data access can be developed, using the relational database support for these databases and something else (probably more specific to each NoSQL-Database, because AFAIK they are even less compatible to each other than relational DBs) for NoSQL-DBs. Cheers, - Daniel
Mar 29 2011
prev sibling parent "Masahiro Nakagawa" <repeatedly gmail.com> writes:
On Sun, 27 Mar 2011 07:11:01 +0900, David Nadlinger <see klickverbot.at>=
  =

wrote:

 On 3/26/11 12:44 AM, Masahiro Nakagawa wrote:
 Currently, many databases exist.

 * SQL based: MySQL, PostgresSQL, SQLite, etc..
 * KVS: Cassandra, HBase, Kumofs, Redis, etc...
 * Document Oriented: MongoDB, CouchDB, etc...

 I think next Database API should support these types.

I don't think this is the way to go, as there is a huge conceptional =

 difference between relational database systems and =C2=BBNoSQL=C2=AB s=

 as, say, CouchDB or Cassandra.

It's an open question. Ruby / Rails has Arel (note that Arel is data access abstraction, not database access abstraction. But Database access is a part of data = access). Haskell has Database.Persistent. I think other languages will support similar library in the future.
 How would such a unified API look like?

I don't have a detail. I already implemented it if I had a detailed design. In my D tasks, database related task is low priority...
Mar 28 2011