digitalmars.D.announce - Abstract Database Interface
- BLM768 (59/59) Oct 27 2012 I've recently been working with Ruby's ActiveRecord as part of my
- Jacob Carlborg (25/75) Oct 28 2012 Looking at the API used in this example it would say that it's not very
- BLM768 (13/36) Oct 29 2012 It's definitely not ActiveRecord, but my goal is just to take
- Jacob Carlborg (4/15) Oct 29 2012 My point was just that you removed the key features and soul of
- BLM768 (14/17) Oct 29 2012 That's a good point. I haven't had any experience with other ORM
- Jacob Carlborg (9/20) Oct 29 2012 You can have a look at DataMapper. That's also for Ruby but it's not
- BLM768 (3/9) Oct 29 2012 Those libraries definitely look interesting. I should probably
- Jacob Carlborg (13/23) Oct 29 2012 If I recall correctly, Squeryl use Scala AST macros to support a query
- BLM768 (10/20) Oct 29 2012 If you make x some fancy wrapper type containing more fancy
- Jacob Carlborg (7/16) Oct 30 2012 It can probably all be handled at compile time. The problem with this
- Philippe Sigaud (4/13) Oct 30 2012 && and || can be replaced by & and |, so there is a workaround.
- Kapps (10/34) Oct 30 2012 There was a pull request for __traits(codeof, func) that would
- Jacob Carlborg (6/13) Oct 30 2012 How would that work in this case, the code need to compile? I mean, even...
- Jacob Carlborg (11/14) Oct 30 2012 How does that work with operator precedence? There's a plugin for
- Philippe Sigaud (5/8) Oct 30 2012 I think the problem would the same here. Of course, to know D operator
- Timon Gehr (23/31) Oct 30 2012 From higher to lower, where relational ops are unordered with respect
- Philippe Sigaud (5/27) Oct 31 2012 Thanks a lot Timon!
- Timon Gehr (2/30) Dec 23 2012 http://wiki.dlang.org/DWiki:Operator_precedence
- Philippe Sigaud (5/13) Dec 24 2012 Seems good. I remember levels 6-8 from the online grammar. I guess it fe...
- BLM768 (11/11) Dec 24 2012 I was kind of shocked to find this thread at the top of the list
- Rory McGuire (2/51) Oct 28 2012
- Robik (5/65) Oct 28 2012 I am working on similar project, named SQLd[1]. If you are
- BLM768 (6/10) Oct 29 2012 That might be a good idea. I haven't done much for supporting
- Marco Leise (13/26) Nov 06 2012 I would use such an ORM library, too. It is definitely worth
I've recently been working with Ruby's ActiveRecord as part of my job, and I realized that D was powerful enough to make a similar abstraction layer. I've been playing with the idea for a little while, and I've put up some code at https://github.com/blm768/adbi. It isn't nearly as comprehensive as ActiveRecord, but it captures the basic idea of representing database structures as objects/structs. Using it is something like this: module main; import std.stdio; import adbi.model; import adbi.sqlite3.database; struct Test { mixin Model!"test"; const(char)[] name; double num; mixin reference!("test2", "test2", Test2); } struct Test2 { mixin Model!"test2"; int value; } int main(string[] args) { auto db = new Sqlite3Database("test.db"); auto q = db.query("SELECT * FROM test;"); Test.updateSchema(db); Test2.updateSchema(db); auto range = ModelRange!Test(q); foreach(value; range) { writeln(value); } auto q2 = db.query("SELECT * FROM test, test2 WHERE test2_id = test2.id"); auto r2 = ModelRange!(Join!(Test, Test2))(q2); foreach(j; r2) { writeln(j); } return 0; } This code prints out every entry in the "test" table, then prints the results of a join on the "test" and "test2" tables. The calls to updateSchema() set up some static members of Test and Test2; after these calls, the library does not perform any operations with the column names, which should make retrieving a record noticeably faster than in a system like ActiveRecord. The downside is that these functions must be called every time the database schema changes in a way that affects column order, but that should happen very rarely, if ever, in a typical application. The code is far from complete, but it's an interesting toy and might actually be useful for simple applications once some of the issues are ironed out.
Oct 27 2012
On 2012-10-28 02:31, BLM768 wrote:I've recently been working with Ruby's ActiveRecord as part of my job, and I realized that D was powerful enough to make a similar abstraction layer. I've been playing with the idea for a little while, and I've put up some code at https://github.com/blm768/adbi. It isn't nearly as comprehensive as ActiveRecord, but it captures the basic idea of representing database structures as objects/structs. Using it is something like this: module main; import std.stdio; import adbi.model; import adbi.sqlite3.database; struct Test { mixin Model!"test"; const(char)[] name; double num; mixin reference!("test2", "test2", Test2); } struct Test2 { mixin Model!"test2"; int value; } int main(string[] args) { auto db = new Sqlite3Database("test.db"); auto q = db.query("SELECT * FROM test;"); Test.updateSchema(db); Test2.updateSchema(db); auto range = ModelRange!Test(q); foreach(value; range) { writeln(value); } auto q2 = db.query("SELECT * FROM test, test2 WHERE test2_id = test2.id"); auto r2 = ModelRange!(Join!(Test, Test2))(q2); foreach(j; r2) { writeln(j); } return 0; } This code prints out every entry in the "test" table, then prints the results of a join on the "test" and "test2" tables. The calls to updateSchema() set up some static members of Test and Test2; after these calls, the library does not perform any operations with the column names, which should make retrieving a record noticeably faster than in a system like ActiveRecord. The downside is that these functions must be called every time the database schema changes in a way that affects column order, but that should happen very rarely, if ever, in a typical application. The code is far from complete, but it's an interesting toy and might actually be useful for simple applications once some of the issues are ironed out.Looking at the API used in this example it would say that it's not very interesting and not very ActiveRecrod like. I think this looks more interesting and more like ActiveRecrod: class Person : Model { } void main () { auto p = new Person; p.name = "John Doe"; p.save(); p = Person.where!(x => x.name == "John Doe"); } But when you start to use associative it won't be as nice looking as ActiveRecord due to the not so nice mixin syntax. What we need is AST macros and user defined attributes/annotations. With that, associations could potentially look like this: class Foo : Model {} class Person : Model { hasMany Foo; } -- /Jacob Carlborg
Oct 28 2012
Looking at the API used in this example it would say that it's not very interesting and not very ActiveRecrod like. I think this looks more interesting and more like ActiveRecrod: class Person : Model { } void main () { auto p = new Person; p.name = "John Doe"; p.save(); p = Person.where!(x => x.name == "John Doe"); } But when you start to use associative it won't be as nice looking as ActiveRecord due to the not so nice mixin syntax. What we need is AST macros and user defined attributes/annotations. With that, associations could potentially look like this: class Foo : Model {} class Person : Model { hasMany Foo; }It's definitely not ActiveRecord, but my goal is just to take some inspiration from it, not to duplicate it. I'm very concerned about efficiency, which is why I'm using structs, and I like hard-coding the fields into the structure so there's some documentation of what the record is supposed to hold and so the compiler can optimize it more heavily. It will probably be a little less pretty, but it'll work, and that's what really matters. At some point, I might implement an interface to generate SQL queries with function calls, but for now, just manually writing the queries really isn't hard, and it provides a significant speed boost for systems like Sqlite that compile queries down to bytecode because it's easier to reuse the query object.
Oct 29 2012
It's definitely not ActiveRecord, but my goal is just to take some inspiration from it, not to duplicate it. I'm very concerned about efficiency, which is why I'm using structs, and I like hard-coding the fields into the structure so there's some documentation of what the record is supposed to hold and so the compiler can optimize it more heavily. It will probably be a little less pretty, but it'll work, and that's what really matters. At some point, I might implement an interface to generate SQL queries with function calls, but for now, just manually writing the queries really isn't hard, and it provides a significant speed boost for systems like Sqlite that compile queries down to bytecode because it's easier to reuse the query object.My point was just that you removed the key features and soul of ActiveRecord. Without these features it's just like any other ORM library. -- /Jacob Carlborg
Oct 29 2012
My point was just that you removed the key features and soul of ActiveRecord. Without these features it's just like any other ORM library.That's a good point. I haven't had any experience with other ORM libraries, so ActiveRecord was the closest thing that came to mind. I definitely do want to eventually capture some of ActiveRecord's features, but probably not all of them. I feel like the solution should be implemented in a way that fits well with a statically typed language, so I'll definitely have to drop some of the features. It won't be quite as nice to use, but it will be simpler in dome ways, which is one of my primary goals as a developer. Tools like ActiveRecord are more fun to use, but thinking of all the hash table lookups makes me cringe. :) If and when the library matures, though, I might think about adding some more ActiveRecord-like features if enough people miss them.
Oct 29 2012
On 2012-10-29 15:42, BLM768 wrote:That's a good point. I haven't had any experience with other ORM libraries, so ActiveRecord was the closest thing that came to mind. I definitely do want to eventually capture some of ActiveRecord's features, but probably not all of them. I feel like the solution should be implemented in a way that fits well with a statically typed language, so I'll definitely have to drop some of the features. It won't be quite as nice to use, but it will be simpler in dome ways, which is one of my primary goals as a developer. Tools like ActiveRecord are more fun to use, but thinking of all the hash table lookups makes me cringe. :) If and when the library matures, though, I might think about adding some more ActiveRecord-like features if enough people miss them.You can have a look at DataMapper. That's also for Ruby but it's not specific for SQL, if I recall correctly. Have a look at some ORM library written in Scala, I would guess they can be quite innovative and it's statically typed. http://squeryl.org/index.html http://datamapper.org/ -- /Jacob Carlborg
Oct 29 2012
You can have a look at DataMapper. That's also for Ruby but it's not specific for SQL, if I recall correctly. Have a look at some ORM library written in Scala, I would guess they can be quite innovative and it's statically typed. http://squeryl.org/index.html http://datamapper.org/Those libraries definitely look interesting. I should probably consider some type of NoSQL database support... Thanks for the links!
Oct 29 2012
On 2012-10-29 18:43, BLM768 wrote:If I recall correctly, Squeryl use Scala AST macros to support a query syntax, that in D would look, as below: class Person : Model { } void main () { auto p = new Person; p.name = "John Doe"; p.save(); p = Person.where!(x => x.name == "John Doe"); } -- /Jacob CarlborgYou can have a look at DataMapper. That's also for Ruby but it's not specific for SQL, if I recall correctly. Have a look at some ORM library written in Scala, I would guess they can be quite innovative and it's statically typed. http://squeryl.org/index.html http://datamapper.org/Those libraries definitely look interesting. I should probably consider some type of NoSQL database support...
Oct 29 2012
If I recall correctly, Squeryl use Scala AST macros to support a query syntax, that in D would look, as below: class Person : Model { } void main () { auto p = new Person; p.name = "John Doe"; p.save(); p = Person.where!(x => x.name == "John Doe"); }If you make x some fancy wrapper type containing more fancy wrapper types with overloaded equality operators that return some sort of Expression class instead of a boolean, you might actually be able to get this to work with only D's current features. However, that would kind of destroy the hope of efficiency. :) What might be nice is a database written in D that completely eschews SQL in favor of a native API. I might have to play with that eventually, but I'll probably give it a while because it would be a huge project, and, like most people, I'm under time constraints. :)
Oct 29 2012
On 2012-10-30 04:22, BLM768 wrote:If you make x some fancy wrapper type containing more fancy wrapper types with overloaded equality operators that return some sort of Expression class instead of a boolean, you might actually be able to get this to work with only D's current features. However, that would kind of destroy the hope of efficiency. :)It can probably all be handled at compile time. The problem with this that you cannot overload the following operators: &&, ||, != and probably some other useful operators.What might be nice is a database written in D that completely eschews SQL in favor of a native API. I might have to play with that eventually, but I'll probably give it a while because it would be a huge project, and, like most people, I'm under time constraints. :)Yeah, I know. -- /Jacob Carlborg
Oct 30 2012
On Tue, Oct 30, 2012 at 9:15 AM, Jacob Carlborg <doob me.com> wrote:On 2012-10-30 04:22, BLM768 wrote:&& and || can be replaced by & and |, so there is a workaround. I feel much more limited by != and, even more problematic, !. Maybe unary - could be used in lieu of !.If you make x some fancy wrapper type containing more fancy wrapper types with overloaded equality operators that return some sort of Expression class instead of a boolean, you might actually be able to get this to work with only D's current features. However, that would kind of destroy the hope of efficiency. :)It can probably all be handled at compile time. The problem with this that you cannot overload the following operators: &&, ||, != and probably some other useful operators.
Oct 30 2012
On Tuesday, 30 October 2012 at 10:01:06 UTC, Philippe Sigaud wrote:On Tue, Oct 30, 2012 at 9:15 AM, Jacob Carlborg <doob me.com> wrote:There was a pull request for __traits(codeof, func) that would return the code for a symbol including lambda methods. It would probably be easier to have something like that for getting the LINQ does it) than using sketchy hacks that go against natural language feel. Though it wouldn't be particularly easy to get that in to the compiler apparently due to AST rewriting issues. https://github.com/D-Programming-Language/dmd/pull/953On 2012-10-30 04:22, BLM768 wrote:&& and || can be replaced by & and |, so there is a workaround. I feel much more limited by != and, even more problematic, !. Maybe unary - could be used in lieu of !.If you make x some fancy wrapper type containing more fancy wrapper types with overloaded equality operators that return some sort of Expression class instead of a boolean, you might actually be able to get this to work with only D's current features. However, that would kind of destroy the hope of efficiency. :)It can probably all be handled at compile time. The problem with this that you cannot overload the following operators: &&, ||, != and probably some other useful operators.
Oct 30 2012
On 2012-10-30 13:04, Kapps wrote:There was a pull request for __traits(codeof, func) that would return the code for a symbol including lambda methods. It would probably be easier to have something like that for getting the AST and then using sketchy hacks that go against natural language feel. Though it wouldn't be particularly easy to get that in to the compiler apparently due to AST rewriting issues.How would that work in this case, the code need to compile? I mean, even if you can get the syntax of a function, process it correctly and generate SQL from it, the function still need to compile. -- /Jacob Carlborg
Oct 30 2012
On 2012-10-30 10:59, Philippe Sigaud wrote:&& and || can be replaced by & and |, so there is a workaround. I feel much more limited by != and, even more problematic, !. Maybe unary - could be used in lieu of !.How does that work with operator precedence? There's a plugin for ActiveRecord, called Squeel, that allows you to do something like this: Person.where do |q| (q.name == "asd") & (q.address == "foo") end But because of the operator precedence in Ruby you need to wrap every comparison in parentheses, not very pretty. -- /Jacob Carlborg
Oct 30 2012
On Tue, Oct 30, 2012 at 3:44 PM, Jacob Carlborg <doob me.com> wrote:How does that work with operator precedence?(...)But because of the operator precedence in Ruby you need to wrap every comparison in parentheses, not very pretty.I think the problem would the same here. Of course, to know D operator precedence, you have to dig into the grammar, since there is no handy table to give you that info :)
Oct 30 2012
On 10/30/2012 04:47 PM, Philippe Sigaud wrote:On Tue, Oct 30, 2012 at 3:44 PM, Jacob Carlborg <doob me.com> wrote:From higher to lower, where relational ops are unordered with respect to bitwise ops (this is the reason comparisons would have to be wrapped in parentheses in D as well): ! => (not a real operator, occurs twice this is binding power to the left) . ++ -- ( [ ^^ (right-associative) & ++ -- * - + ! ~ (prefix) * / % + - ~ << >> >>> == != > < >= <= !> !< !>= !<= <> !<> <>= !<>= in !in is !is & ^ | && || ? (right-associative) /= &= |= -= += <<= >>= >>>= = *= %= ^= ^^= ~= (right-associative) => (not a real operator, occurs twice, this is binding power to the right) , .. (not a real operator)How does that work with operator precedence?(...)But because of the operator precedence in Ruby you need to wrap every comparison in parentheses, not very pretty.I think the problem would the same here. Of course, to know D operator precedence, you have to dig into the grammar, since there is no handy table to give you that info :)
Oct 30 2012
Timon:From higher to lower, where relational ops are unordered with respect to bitwise ops (this is the reason comparisons would have to be wrapped in parentheses in D as well): ! => (not a real operator, occurs twice this is binding power to the left) . ++ -- ( [ ^^ (right-associative) & ++ -- * - + ! ~ (prefix) * / % + - ~ << >> >>> == != > < >= <= !> !< !>= !<= <> !<> <>= !<>= in !in is !is & ^ | && || ? (right-associative) /= &= |= -= += <<= >>= >>>= = *= %= ^= ^^= ~= (right-associative) => (not a real operator, occurs twice, this is binding power to the right) , .. (not a real operator)Thanks a lot Timon! => is indeed a strange beast. Any possibility to put that on a wiki page or the dlang.org on operator overloading?
Oct 31 2012
On 10/31/2012 05:54 PM, Philippe Sigaud wrote:Timon:http://wiki.dlang.org/DWiki:Operator_precedenceFrom higher to lower, where relational ops are unordered with respect to bitwise ops (this is the reason comparisons would have to be wrapped in parentheses in D as well): ! => (not a real operator, occurs twice this is binding power to the left) . ++ -- ( [ ^^ (right-associative) & ++ -- * - + ! ~ (prefix) * / % + - ~ << >> >>> == != > < >= <= !> !< !>= !<= <> !<> <>= !<>= in !in is !is & ^ | && || ? (right-associative) /= &= |= -= += <<= >>= >>>= = *= %= ^= ^^= ~= (right-associative) => (not a real operator, occurs twice, this is binding power to the right) , .. (not a real operator)Thanks a lot Timon! => is indeed a strange beast. Any possibility to put that on a wiki page or the dlang.org on operator overloading?
Dec 23 2012
On Sun, Dec 23, 2012 at 7:37 PM, Timon Gehr <timon.gehr gmx.ch> wrote:On 10/31/2012 05:54 PM, Philippe Sigaud wrote:Seems good. I remember levels 6-8 from the online grammar. I guess it feels natural when we use them. The wiki has a much better appearance, I'll try to participate and fill some parts.http://wiki.dlang.org/DWiki:**Operator_precedence<http://wiki.dlang.org/DWiki:Operator_precedence>=> is indeed a strange beast.Any possibility to put that on a wiki page or the dlang.org on operator overloading?
Dec 24 2012
I was kind of shocked to find this thread at the top of the list again; I thought it had died. :) This is a very interesting time for the thread to revive because I was just thinking about the project this morning... Now that my interest has revived a bit, I'm trying to figure out how to get SQLite working on Windows; the only real options I've seen so far are to compile SQLite with DMC or to use GDC instead of DMD. Creating a .lib from sqlite3.dll looks like too much of a pain. I'm also thinking about adding a PosgreSQL driver; I'll probably use code from SQLd for that.
Dec 24 2012
I like, nice work. On 28 Oct 2012 02:35, "BLM768" <blm768 gmail.com> wrote:I've recently been working with Ruby's ActiveRecord as part of my job, and I realized that D was powerful enough to make a similar abstraction layer. I've been playing with the idea for a little while, and I've put up some code at https://github.com/blm768/adbi**. It isn't nearly as comprehensive as ActiveRecord, but it captures the basic idea of representing database structures as objects/structs. Using it is something like this: module main; import std.stdio; import adbi.model; import adbi.sqlite3.database; struct Test { mixin Model!"test"; const(char)[] name; double num; mixin reference!("test2", "test2", Test2); } struct Test2 { mixin Model!"test2"; int value; } int main(string[] args) { auto db = new Sqlite3Database("test.db"); auto q = db.query("SELECT * FROM test;"); Test.updateSchema(db); Test2.updateSchema(db); auto range = ModelRange!Test(q); foreach(value; range) { writeln(value); } auto q2 = db.query("SELECT * FROM test, test2 WHERE test2_id = test2.id"); auto r2 = ModelRange!(Join!(Test, Test2))(q2); foreach(j; r2) { writeln(j); } return 0; } This code prints out every entry in the "test" table, then prints the results of a join on the "test" and "test2" tables. The calls to updateSchema() set up some static members of Test and Test2; after these calls, the library does not perform any operations with the column names, which should make retrieving a record noticeably faster than in a system like ActiveRecord. The downside is that these functions must be called every time the database schema changes in a way that affects column order, but that should happen very rarely, if ever, in a typical application. The code is far from complete, but it's an interesting toy and might actually be useful for simple applications once some of the issues are ironed out.
Oct 28 2012
On Sunday, 28 October 2012 at 00:31:49 UTC, BLM768 wrote:I've recently been working with Ruby's ActiveRecord as part of my job, and I realized that D was powerful enough to make a similar abstraction layer. I've been playing with the idea for a little while, and I've put up some code at https://github.com/blm768/adbi. It isn't nearly as comprehensive as ActiveRecord, but it captures the basic idea of representing database structures as objects/structs. Using it is something like this: module main; import std.stdio; import adbi.model; import adbi.sqlite3.database; struct Test { mixin Model!"test"; const(char)[] name; double num; mixin reference!("test2", "test2", Test2); } struct Test2 { mixin Model!"test2"; int value; } int main(string[] args) { auto db = new Sqlite3Database("test.db"); auto q = db.query("SELECT * FROM test;"); Test.updateSchema(db); Test2.updateSchema(db); auto range = ModelRange!Test(q); foreach(value; range) { writeln(value); } auto q2 = db.query("SELECT * FROM test, test2 WHERE test2_id = test2.id"); auto r2 = ModelRange!(Join!(Test, Test2))(q2); foreach(j; r2) { writeln(j); } return 0; } This code prints out every entry in the "test" table, then prints the results of a join on the "test" and "test2" tables. The calls to updateSchema() set up some static members of Test and Test2; after these calls, the library does not perform any operations with the column names, which should make retrieving a record noticeably faster than in a system like ActiveRecord. The downside is that these functions must be called every time the database schema changes in a way that affects column order, but that should happen very rarely, if ever, in a typical application. The code is far from complete, but it's an interesting toy and might actually be useful for simple applications once some of the issues are ironed out.I am working on similar project, named SQLd[1]. If you are interested, we can join forces and work togheder :) IRC Nick: Robik [1]: http://github.com/robik/SQLd
Oct 28 2012
I am working on similar project, named SQLd[1]. If you are interested, we can join forces and work togheder :) IRC Nick: Robik [1]: http://github.com/robik/SQLdThat might be a good idea. I haven't done much for supporting different databases, so getting more backend support would be quite nice. What might work well is for me to just refactor my code to sit on top of your existing database classes. So far, I've really just been playing around, but if people show enough interest, I'd like to play with the idea a while longer. :)
Oct 29 2012
Am Mon, 29 Oct 2012 08:50:19 +0100 schrieb "BLM768" <blm768 gmail.com>:I would use such an ORM library, too. It is definitely worth the effort. Currently I use something that just helps me a bit with fetching and updating entries in tables, but goes in the same direction of using only structs for table rows. Here are some database bindings from wiki4d that others have written: http://mleise.dnsd.info/#DatabaseDrivers Some are for D1 though. But now that you are team you may want to look at those bindings for some inspiration, so you don't get locked in on SQLite3 too much :p -- MarcoI am working on similar project, named SQLd[1]. If you are interested, we can join forces and work togheder :) IRC Nick: Robik [1]: http://github.com/robik/SQLdThat might be a good idea. I haven't done much for supporting different databases, so getting more backend support would be quite nice. What might work well is for me to just refactor my code to sit on top of your existing database classes. So far, I've really just been playing around, but if people show enough interest, I'd like to play with the idea a while longer. :)
Nov 06 2012