www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Right after allocators: containers or database connectivity?

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
My work on allocators takes the last turn before the straight line. I've 
arranged with Dicebot to overlap the review period with finalizing 
details so I can act on feedback quickly.

After that I'm ready for some major library work, and I had two things 
in mind.

One would be a good pass of std.container, in particular (a) a design 
review with the DbI glasses on; (b) better documentation - sadly it 
seems to me so inadequate as to make containers themselves unusable; (c) 
investigate use of UFCS - std.container's design predates UFCS yet is a 
perfect fit for it, and most likely other cool language improvements 
we've added since.

The other would be database connectivity. Erik Smith has shown some cool 
ideas at DConf, and I encourage him to continue working on them, but it 
seems to me this is an area where more angles mean more connectivity 
options.

For database connectivity I'm thinking of using ODBC. What I see is that 
on all major platforms, vendors offer mature, good quality ODBC drivers, 
and most programs that have anything to do with databases offer ODBC 
connectivity. So connecting with ODBC means the individual database 
drivers are already there; no need to waste effort on creating drivers 
for each (or asking vendors to, which we can't afford).

So I gave myself ten minutes the other night just before I went to sleep 
to see if I can get an ODBC rig on my OSX machine starting from 
absolutely nothing. I got http://www.odbcmanager.net but then got 
confused about where to find some dumb driver (text, csv) and gave up.

Last night I gave myself another ten minutes, and lo and behold I got up 
and running. Got a demo CSV driver from 
http://www.actualtech.com/product_access.php (which also supports Access 
and others). Then I headed to 
http://www.easysoft.com/developer/languages/c/odbc_tutorial.html and was 
able to run a simple ODBC application that lists the available drivers. 
Nice!

It's trivial work to convert the C headers to D declarations. Then it's 
straight library design to offer convenient libraries on top of the 
comprehensive but pedestrian ODBC C API. Then, voilà - we'll have 
database connectivity for all databases out there!

Please help me choose what to work on next.


Andrei
Jun 09 2015
next sibling parent reply "Liam McSherry" <mcsherry.liam gmail.com> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 help me choose what to work on next.


 Andrei
I think containers would be the better option. D sorely needs to have basics like (de)queues and stacks in the standard library, as well as any other popular and useful containers.
Jun 09 2015
next sibling parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Tuesday, 9 June 2015 at 17:12:53 UTC, Liam McSherry wrote:
 I think containers would be the better option. D sorely needs 
 to have basics like (de)queues and stacks in the standard 
 library, as well as any other popular and useful containers.
And `set`.
Jun 09 2015
next sibling parent reply "kinke" <noone nowhere.com> writes:
On Tuesday, 9 June 2015 at 17:15:46 UTC, Dennis Ritchie wrote:
 On Tuesday, 9 June 2015 at 17:12:53 UTC, Liam McSherry wrote:
 I think containers would be the better option. D sorely needs 
 to have basics like (de)queues and stacks in the standard 
 library, as well as any other popular and useful containers.
And `set`.
+1, I was just about to post this too. ;)
Jun 09 2015
parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Tuesday, 9 June 2015 at 17:27:34 UTC, kinke wrote:
 On Tuesday, 9 June 2015 at 17:15:46 UTC, Dennis Ritchie wrote:
 And `set`.
+1, I was just about to post this too. ;)
You already ahead in DIP80: http://wiki.dlang.org/DIP80
Jun 09 2015
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 9 June 2015 at 17:15:46 UTC, Dennis Ritchie wrote:
 On Tuesday, 9 June 2015 at 17:12:53 UTC, Liam McSherry wrote:
 I think containers would be the better option. D sorely needs 
 to have basics like (de)queues and stacks in the standard 
 library, as well as any other popular and useful containers.
And `set`.
We have it via RedBlackTree; it's just ugly to use, since you have to keep passing in tuples with values that are ignored. But with a convenience wrapper around RedBlackTree, that's easily fixed. So, we _do_ have it. It's just nowhere near as user-friendly as it should be. - Jonathan M Davis
Jun 09 2015
prev sibling parent Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 09 Jun 2015 17:12:51 +0000
Liam McSherry via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
 wrote:
 help me choose what to work on next.


 Andrei
I think containers would be the better option. D sorely needs to have basics like (de)queues and stacks in the standard library, as well as any other popular and useful containers.
+1000
Jun 09 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-09 19:05, Andrei Alexandrescu wrote:
 My work on allocators takes the last turn before the straight line. I've
 arranged with Dicebot to overlap the review period with finalizing
 details so I can act on feedback quickly.

 After that I'm ready for some major library work, and I had two things
 in mind.

 One would be a good pass of std.container, in particular (a) a design
 review with the DbI glasses on; (b) better documentation - sadly it
 seems to me so inadequate as to make containers themselves unusable; (c)
 investigate use of UFCS - std.container's design predates UFCS yet is a
 perfect fit for it, and most likely other cool language improvements
 we've added since.

 The other would be database connectivity.
I vote for databases. I also vote for an interface that is independent of ODBC, but ODBC could be one of the drivers that implements this interface. I know that there's a native implementation of the MySQL protocol on code.dlang.org. -- /Jacob Carlborg
Jun 09 2015
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/09/2015 01:53 PM, Jacob Carlborg wrote:
 I vote for databases. I also vote for an interface that is independent
 of ODBC, but ODBC could be one of the drivers that implements this
 interface. I know that there's a native implementation of the MySQL
 protocol on code.dlang.org.
This is kind of my thought, too. I haven't felt constrained by the lack of containers beyond what's built-in. And if I do, I can look into dcollections. It's arguably an optimization issue anyway. YES, Granted, algorithmic optimization is THE BIGGEST optimization by far, but aside from maybe "big data" domains it doesn't actually enable you to do things you just plain couldn't do at all otherwise. Databases, OTOH, are something where you either can do it, or you just plain CAN'T. And with D it's currently a bit more on the "can't" side, especially if you don't want your app tied to just one DBMS. And unlike containers, the state of D databases actually HAS been an issue for me. That said, tossing an ODBC wrapper in phobos doesn't really solve the DB issue. It would certainly be GOOD to have a good ODBC lib for D, no doubt, but: 1. There is apparently question as to whether it belongs in Phobos. 2. There's also question about async I/O. A *LOT* of DB users are also going to be Vibe.d users, so we cannot afford to have problems in this area. Perhaps what we need to look at is getting Vibe's async I/O (or at least some low-level portion of it, if that even makes sense?) into Phobos somehow? 3. As useful as ODBC may be, it still doesn't eliminate the need for direct drivers. 4. What's REALLY needed regarding DBs is a good, well-designed low-level interface capable of handling both ODBC and individual drivers. ORMs should be something separate, built on top of this later. This low-level part might either include NoSQL (if reasonable to do so) or leave NoSQL up to a separate interface (if need be).
Jun 11 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/11/15 10:59 AM, Nick Sabalausky wrote:
 2. There's also question about async I/O. A *LOT* of DB users are also
 going to be Vibe.d users, so we cannot afford to have problems in this
 area. Perhaps what we need to look at is getting Vibe's async I/O (or at
 least some low-level portion of it, if that even makes sense?) into
 Phobos somehow?
BTW I'm unclear about the ODBC async support on OSX and Linux, and Internet searches are not making that crystal clear. Does anyone know? -- Andrei
Jun 11 2015
prev sibling parent "Erik Smith" <erik cruiserhouse.com> writes:
I’m actively working on the low level interface and I’m making 
good progress.  My goal is to have a high quality interface 
definition with working reference drivers for top tier databases. 
  The current references drivers are ODBC, sqlite, mysql, and 
Oracle.

I think it’s important to support ODBC since it covers all bases, 
but I agree that ODBC alone is not sufficient.  The main reason 
is that most application developers coming from other stacks will 
be expecting direct drivers and are not likely to be happy 
dealing with the added complications of ODBC.   However, I think 
both are achievable in the near term under a common interface.

On the async question, the native interfaces are traditionally 
synchronous and that’s my initial focus.   I will need to look at 
which native interfaces actually support it and see how that can 
be mapped into the generic interface, but synchronous needs to be 
supported.

There is a known issue with RefCounted that needs to be 
addressed.  There is a apparently a race condition with the 
reference counter when RefCounted is within a class.  Nearly all 
of the types in my design are ref counted structs and there are 
use cases for embedding those objects in classes.  There was 
discussion at the conference on this and I believe the preferred 
solution is for the GC have some thread affinity on collection.
Jun 11 2015
prev sibling next sibling parent reply "luminousone" <rd.hunt gmail.com> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 My work on allocators takes the last turn before the straight 
 line. I've arranged with Dicebot to overlap the review period 
 with finalizing details so I can act on feedback quickly.

 After that I'm ready for some major library work, and I had two 
 things in mind.

 One would be a good pass of std.container, in particular (a) a 
 design review with the DbI glasses on; (b) better documentation 
 - sadly it seems to me so inadequate as to make containers 
 themselves unusable; (c) investigate use of UFCS - 
 std.container's design predates UFCS yet is a perfect fit for 
 it, and most likely other cool language improvements we've 
 added since.

 The other would be database connectivity. Erik Smith has shown 
 some cool ideas at DConf, and I encourage him to continue 
 working on them, but it seems to me this is an area where more 
 angles mean more connectivity options.

 For database connectivity I'm thinking of using ODBC. What I 
 see is that on all major platforms, vendors offer mature, good 
 quality ODBC drivers, and most programs that have anything to 
 do with databases offer ODBC connectivity. So connecting with 
 ODBC means the individual database drivers are already there; 
 no need to waste effort on creating drivers for each (or asking 
 vendors to, which we can't afford).

 So I gave myself ten minutes the other night just before I went 
 to sleep to see if I can get an ODBC rig on my OSX machine 
 starting from absolutely nothing. I got 
 http://www.odbcmanager.net but then got confused about where to 
 find some dumb driver (text, csv) and gave up.

 Last night I gave myself another ten minutes, and lo and behold 
 I got up and running. Got a demo CSV driver from 
 http://www.actualtech.com/product_access.php (which also 
 supports Access and others). Then I headed to 
 http://www.easysoft.com/developer/languages/c/odbc_tutorial.html 
 and was able to run a simple ODBC application that lists the 
 available drivers. Nice!

 It's trivial work to convert the C headers to D declarations. 
 Then it's straight library design to offer convenient libraries 
 on top of the comprehensive but pedestrian ODBC C API. Then, 
 voilà - we'll have database connectivity for all databases out 
 there!

 Please help me choose what to work on next.


 Andrei
I would think a good database lib would depend on a good container lib, So IMO, the order is self evident. 1000+ to containers.
Jun 09 2015
next sibling parent Charles Hixson via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 06/09/2015 10:56 AM, luminousone via Digitalmars-d wrote:
 On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu wrote:
 My work on allocators takes the last turn before the straight line. 
 I've arranged with Dicebot to overlap the review period with 
 finalizing details so I can act on feedback quickly.

 After that I'm ready for some major library work, and I had two 
 things in mind.

 One would be a good pass of std.container, in particular (a) a design 
 review with the DbI glasses on; (b) better documentation - sadly it 
 seems to me so inadequate as to make containers themselves unusable; 
 (c) investigate use of UFCS - std.container's design predates UFCS 
 yet is a perfect fit for it, and most likely other cool language 
 improvements we've added since.

 The other would be database connectivity. Erik Smith has shown some 
 cool ideas at DConf, and I encourage him to continue working on them, 
 but it seems to me this is an area where more angles mean more 
 connectivity options.

 For database connectivity I'm thinking of using ODBC. What I see is 
 that on all major platforms, vendors offer mature, good quality ODBC 
 drivers, and most programs that have anything to do with databases 
 offer ODBC connectivity. So connecting with ODBC means the individual 
 database drivers are already there; no need to waste effort on 
 creating drivers for each (or asking vendors to, which we can't afford).

 So I gave myself ten minutes the other night just before I went to 
 sleep to see if I can get an ODBC rig on my OSX machine starting from 
 absolutely nothing. I got http://www.odbcmanager.net but then got 
 confused about where to find some dumb driver (text, csv) and gave up.

 Last night I gave myself another ten minutes, and lo and behold I got 
 up and running. Got a demo CSV driver from 
 http://www.actualtech.com/product_access.php (which also supports 
 Access and others). Then I headed to 
 http://www.easysoft.com/developer/languages/c/odbc_tutorial.html and 
 was able to run a simple ODBC application that lists the available 
 drivers. Nice!

 It's trivial work to convert the C headers to D declarations. Then 
 it's straight library design to offer convenient libraries on top of 
 the comprehensive but pedestrian ODBC C API. Then, voilà - we'll have 
 database connectivity for all databases out there!

 Please help me choose what to work on next.


 Andrei
I would think a good database lib would depend on a good container lib, So IMO, the order is self evident. 1000+ to containers.
I'd say what it depends on is good serialization. If you extend containers to include disk based B+Trees then I'd agree with you, otherwise not.
Jun 09 2015
prev sibling parent "Chris" <wendlec tcd.ie> writes:
On Tuesday, 9 June 2015 at 17:56:57 UTC, luminousone wrote:
 On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
 wrote:
 My work on allocators takes the last turn before the straight 
 line. I've arranged with Dicebot to overlap the review period 
 with finalizing details so I can act on feedback quickly.

 After that I'm ready for some major library work, and I had 
 two things in mind.

 One would be a good pass of std.container, in particular (a) a 
 design review with the DbI glasses on; (b) better 
 documentation - sadly it seems to me so inadequate as to make 
 containers themselves unusable; (c) investigate use of UFCS - 
 std.container's design predates UFCS yet is a perfect fit for 
 it, and most likely other cool language improvements we've 
 added since.

 The other would be database connectivity. Erik Smith has shown 
 some cool ideas at DConf, and I encourage him to continue 
 working on them, but it seems to me this is an area where more 
 angles mean more connectivity options.

 For database connectivity I'm thinking of using ODBC. What I 
 see is that on all major platforms, vendors offer mature, good 
 quality ODBC drivers, and most programs that have anything to 
 do with databases offer ODBC connectivity. So connecting with 
 ODBC means the individual database drivers are already there; 
 no need to waste effort on creating drivers for each (or 
 asking vendors to, which we can't afford).

 So I gave myself ten minutes the other night just before I 
 went to sleep to see if I can get an ODBC rig on my OSX 
 machine starting from absolutely nothing. I got 
 http://www.odbcmanager.net but then got confused about where 
 to find some dumb driver (text, csv) and gave up.

 Last night I gave myself another ten minutes, and lo and 
 behold I got up and running. Got a demo CSV driver from 
 http://www.actualtech.com/product_access.php (which also 
 supports Access and others). Then I headed to 
 http://www.easysoft.com/developer/languages/c/odbc_tutorial.html 
 and was able to run a simple ODBC application that lists the 
 available drivers. Nice!

 It's trivial work to convert the C headers to D declarations. 
 Then it's straight library design to offer convenient 
 libraries on top of the comprehensive but pedestrian ODBC C 
 API. Then, voilà - we'll have database connectivity for all 
 databases out there!

 Please help me choose what to work on next.


 Andrei
I would think a good database lib would depend on a good container lib, So IMO, the order is self evident. 1000+ to containers.
Yep. Containers. containers + right allocation strategy => interfacing to data base with the right containers
Jun 10 2015
prev sibling next sibling parent reply Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 09 Jun 2015 10:05:24 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 
 For database connectivity I'm thinking of using ODBC. What I see is that 
 on all major platforms, vendors offer mature, good quality ODBC drivers, 
 and most programs that have anything to do with databases offer ODBC 
 connectivity. So connecting with ODBC means the individual database 
 drivers are already there; no need to waste effort on creating drivers 
 for each (or asking vendors to, which we can't afford).
Having ODBC support in D is definitely important for some kind of applications. But for most applications which works with some (kind of) database it does not scale. We really need individual drivers for each of the most popular databases (even as a C/C++ lib with d binding around it). And I do not think DB conectivity need to be in phobos. So as I allready wrote in another reply I think containers stuff are much more important right now (same as better JSON,XML,BSON...).
Jun 09 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/9/15 10:44 AM, Daniel Kozak via Digitalmars-d wrote:
 On Tue, 09 Jun 2015 10:05:24 -0700
 Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 For database connectivity I'm thinking of using ODBC. What I see is that
 on all major platforms, vendors offer mature, good quality ODBC drivers,
 and most programs that have anything to do with databases offer ODBC
 connectivity. So connecting with ODBC means the individual database
 drivers are already there; no need to waste effort on creating drivers
 for each (or asking vendors to, which we can't afford).
Having ODBC support in D is definitely important for some kind of applications. But for most applications which works with some (kind of) database it does not scale. We really need individual drivers for each of the most popular databases (even as a C/C++ lib with d binding around it).
I derive the exact opposite conclusion from the same facts. * Individual drivers for each database engine: spend effort on designing an API, THEN spend effort on writing or adapting (and then maintaining) one driver per database engine. That has "does not scale" written all over it. * ODBC: design an API on top of ODBC, then ENJOY all the hard work various database engines have put into their drivers. That scales. Andrei
Jun 09 2015
parent reply Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 09 Jun 2015 11:41:37 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 6/9/15 10:44 AM, Daniel Kozak via Digitalmars-d wrote:
 On Tue, 09 Jun 2015 10:05:24 -0700
 Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 For database connectivity I'm thinking of using ODBC. What I see is that
 on all major platforms, vendors offer mature, good quality ODBC drivers,
 and most programs that have anything to do with databases offer ODBC
 connectivity. So connecting with ODBC means the individual database
 drivers are already there; no need to waste effort on creating drivers
 for each (or asking vendors to, which we can't afford).
Having ODBC support in D is definitely important for some kind of applications. But for most applications which works with some (kind of) database it does not scale. We really need individual drivers for each of the most popular databases (even as a C/C++ lib with d binding around it).
I derive the exact opposite conclusion from the same facts. * Individual drivers for each database engine: spend effort on designing an API, THEN spend effort on writing or adapting (and then maintaining) one driver per database engine. That has "does not scale" written all over it. * ODBC: design an API on top of ODBC, then ENJOY all the hard work various database engines have put into their drivers. That scales. Andrei
Yep this is the other side of a coin :), and I agree with that. But I do not belive that performance and features would be on same level as individual connectors.
Jun 09 2015
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 9 June 2015 at 20:17:01 UTC, Daniel Kozak wrote:
 Yep this is the other side of a coin :), and I agree with that. 
 But I do not
 belive that performance and features would be on same level as 
 individual
 connectors.
I confess that what little I've heard about ODBC, it's a terrible idea to use it, but I'm not much of a database guy, so I'm not really in a position to judge. Regardless, it would probably make sense to support both ODBC _and_ the individual drivers. Yes, that's more work, but it provides greater flexibility as well, and we could start with ODBC support so that we have _something_ and add support for individual drivers later so that we get the better performance. Then folks can use whichever they prefer, and we get something working sooner rather than later, even if it takes longer to get it all. - Jonathan M Davis
Jun 09 2015
prev sibling next sibling parent "Kiith-Sa" <kiith ihatesp.am> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu
wrote:
 My work on allocators takes the last turn before the straight 
 line. I've arranged with Dicebot to overlap the review period 
 with finalizing details so I can act on feedback quickly.

 After that I'm ready for some major library work, and I had two 
 things in mind.

 One would be a good pass of std.container, in particular (a) a 
 design review with the DbI glasses on; (b) better documentation 
 - sadly it seems to me so inadequate as to make containers 
 themselves unusable; (c) investigate use of UFCS - 
 std.container's design predates UFCS yet is a perfect fit for 
 it, and most likely other cool language improvements we've 
 added since.

 The other would be database connectivity. Erik Smith has shown 
 some cool ideas at DConf, and I encourage him to continue 
 working on them, but it seems to me this is an area where more 
 angles mean more connectivity options.

 For database connectivity I'm thinking of using ODBC. What I 
 see is that on all major platforms, vendors offer mature, good 
 quality ODBC drivers, and most programs that have anything to 
 do with databases offer ODBC connectivity. So connecting with 
 ODBC means the individual database drivers are already there; 
 no need to waste effort on creating drivers for each (or asking 
 vendors to, which we can't afford).

 So I gave myself ten minutes the other night just before I went 
 to sleep to see if I can get an ODBC rig on my OSX machine 
 starting from absolutely nothing. I got 
 http://www.odbcmanager.net but then got confused about where to 
 find some dumb driver (text, csv) and gave up.

 Last night I gave myself another ten minutes, and lo and behold 
 I got up and running. Got a demo CSV driver from 
 http://www.actualtech.com/product_access.php (which also 
 supports Access and others). Then I headed to 
 http://www.easysoft.com/developer/languages/c/odbc_tutorial.html 
 and was able to run a simple ODBC application that lists the 
 available drivers. Nice!

 It's trivial work to convert the C headers to D declarations. 
 Then it's straight library design to offer convenient libraries 
 on top of the comprehensive but pedestrian ODBC C API. Then, 
 voilà - we'll have database connectivity for all databases out 
 there!

 Please help me choose what to work on next.


 Andrei
Containers. But - we really need those allocators working in actual released Phobos first.
Jun 09 2015
prev sibling next sibling parent reply "Brad Anderson" <eco gnuk.net> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 [snip]
 One would be a good pass of std.container, in particular (a) a 
 design review with the DbI glasses on; (b) better documentation 
 - sadly it seems to me so inadequate as to make containers 
 themselves unusable; (c) investigate use of UFCS - 
 std.container's design predates UFCS yet is a perfect fit for 
 it, and most likely other cool language improvements we've 
 added since.

 [snip]
Containers without a doubt. They are so fundamental to my day to day programming it's amazing that D has gone for so long without a good set of them (dynamic arrays and associative arrays do go a long way though). I might as well share some thoughts on it. Containers that should be in the standard library in order of preference (from my C++ STL and Boost experience primarily): 1. vector/array 2. hash map 3. hash set 4. flat map 5. flat set 6. map 7. set 8. deque 9. stack 10. queue 11. linked list 12. hash multimap 13. hash multiset 14. flat multimap 15. flat multiset 16. multimap 17. multiset 18. priority queue We already have Array for 1 but the interface needs improvement (I seem to recall having trouble with mutating items during iteration or something like that which caused me to go back to dynamic arrays). I don't think I like that it's RefCounted either. r-value references have made C++'s vector much more pleasant to efficiently work with in C++. We also have map in the form of RedBlackTree. I think you might be able to use RedBlackTree for a set too but I haven't tried it. I think RedBlackTree isn't a very good name though. It's long and is an implementation detail. We have hash maps in the form of the built-in associative arrays but something that uses the allocators rather than GC is needed. "Flat" refers to containers that are backed by sorted arrays rather than something like a Red-Black Tree. Insertion speed suffers but in practice, thanks to cache locality and move semantics and fast contiguous memory copying, they are generally much faster than the traditional map backed by something like a red-black tree for the majority of operations. Andrei put them in loki so I'm preaching to the choir here but for anyone that doesn't know about their advantages here's an article: http://lafstern.org/matt/col1.pdf and the Boost container library's rationale: http://www.boost.org/doc/libs/1_56_0/doc/html/container/non_standard_containers.html#container.non_standard_containers.flat_xxx Chandler Carruth spent a lot of time talking about this stuff at CppCon 2014 as well: https://www.youtube.com/watch?v=fHNmRkzxHWs As far as more exotic containers go, I rather like Boost.MultiIndex which lets you have a container with several different characteristics (e.g., a single container where you can look up based on name, based on id, based on insertion order, based on lexicographic order of name, etc.). You basically design the perfect container for your needs ("I need random access and fast lookups and a sorted list and bidirectional lookups between keys and values"). It's the swiss army knife of containers. I don't need it often but when I do I love it. I believe someone said they implemented it for D but I haven't looked into it. Although less versatile, building multi-indexing into most containers could prove useful too. Spitballing an example with plenty of room for improvement: struct A { string name; int id; } Map!(A, ((a, b) => a.name < b.name), ((a, b) => a.id < b.id)) a_map; a_map.insert(A("bob", 7)); assert(a_map.index!0["bob"].id == 7); assert(a_map.index!1[7].name == "bob"); A trie would be nice to have even though I rarely use them. Boost has a stable_vector as well which is basically just an array that uses indirection to keep iterators valid through insertions/deletions. I could see it being useful but I haven't used it.
Jun 09 2015
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 9 June 2015 at 18:49:04 UTC, Brad Anderson wrote:
  We also have map in the form of RedBlackTree. I think you 
 might be able to use RedBlackTree for a set too but I haven't 
 tried it.
You can, but it's a bit of a pain, because you have to keep passing it a tuple where the value gets ignored. We really need a wrapper around it to make it cleaner.
 I think RedBlackTree isn't a very good name though. It's long 
 and is an implementation detail.
The decision was previously made to name the containers after what they actually are rather than what they're used for, and I still agree with that decision. However, I do think that we should have wrappers and aliases where appropriate (e.g. SortedMap could be an alias to RedBlackTree, and SortedSet could be a wrapper around RedBlackTree) in order to provide names which are based on what you use them for (rather than what they are) as folks frequently expectA nd in some cases (like SortedSet), it's pretty much necessary unless you have a lot of code duplication, since the same data structure is used for multiple container types, but the API needs to be slightly different to be truly user-friendly. Regardless, I'm very much in favor of having the core container types be named after what they are rather than what they do, since I think that that often gets lost when it shouldn't. - Jonathan M Davis
Jun 09 2015
next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Tuesday, 9 June 2015 at 22:05:27 UTC, Jonathan M Davis wrote:
 [snip]
 I think RedBlackTree isn't a very good name though. It's long 
 and is an implementation detail.
The decision was previously made to name the containers after what they actually are rather than what they're used for, and I still agree with that decision. However, I do think that we should have wrappers and aliases where appropriate (e.g. SortedMap could be an alias to RedBlackTree, and SortedSet could be a wrapper around RedBlackTree) in order to provide names which are based on what you use them for (rather than what they are) as folks frequently expectA nd in some cases (like SortedSet), it's pretty much necessary unless you have a lot of code duplication, since the same data structure is used for multiple container types, but the API needs to be slightly different to be truly user-friendly.
I suppose an alias would be fine. I can't think of any other standard library that uses the actual data structures as the names. Doing it that way exclusively would be unfriendly to newcomers and downright hostile to people who never have (or have yet to) study computer science. Thinking about it more I actually like the idea now of naming them by the data structure and using aliases. That would also open the possibility of relegating more sophisticated template argument container configuration options (bucket sizes, sparse/dense, etc.) to the data structure to which they apply and letting the container concept aliases be a sane default.
Jun 09 2015
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/9/15 6:05 PM, Jonathan M Davis wrote:
 On Tuesday, 9 June 2015 at 18:49:04 UTC, Brad Anderson wrote:
  We also have map in the form of RedBlackTree. I think you might be
 able to use RedBlackTree for a set too but I haven't tried it.
You can, but it's a bit of a pain, because you have to keep passing it a tuple where the value gets ignored. We really need a wrapper around it to make it cleaner.
Map is a pain. Set is not. The apparatus around RedBlackTree in dcollections to turn it into a map is non-trivial, and is worthy of having in std.container. TreeSet, however, was pretty much straight mapping to implementation calls. -Steve
Jun 11 2015
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 11 June 2015 at 12:47:46 UTC, Steven Schveighoffer 
wrote:
 On 6/9/15 6:05 PM, Jonathan M Davis wrote:
 On Tuesday, 9 June 2015 at 18:49:04 UTC, Brad Anderson wrote:
 We also have map in the form of RedBlackTree. I think you 
 might be
 able to use RedBlackTree for a set too but I haven't tried it.
You can, but it's a bit of a pain, because you have to keep passing it a tuple where the value gets ignored. We really need a wrapper around it to make it cleaner.
Map is a pain. Set is not. The apparatus around RedBlackTree in dcollections to turn it into a map is non-trivial, and is worthy of having in std.container. TreeSet, however, was pretty much straight mapping to implementation calls.
You're right. It's been too long since I used it, and I didn't think it through enough. As I recall, the main problem has to do with all of the cases that you have to pass in a tuple where only the key matters, and for some reason, I was thinking that that was the set case, when it's obviously the map case. The set case could pretty much be an alias if we wanted TreeSet or SortedSet in the library, whereas TreeMap/SortedMap would need to be a wrapper around RedBlackTree. - Jonathan M Davis
Jun 11 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-09 20:49, Brad Anderson wrote:

 1.  vector/array
 2.  hash map
 3.  hash set
 4.  flat map
 5.  flat set
 6.  map
 7.  set
 8.  deque
 9.  stack
 10. queue
 11. linked list
 12. hash multimap
 13. hash multiset
 14. flat multimap
 15. flat multiset
 16. multimap
 17. multiset
 18. priority queue
 I think RedBlackTree isn't a very good name though. It's long and is an
 implementation detail.
In that case you would need to remove "hash" and perhaps "flat" from all containers to be consistent. Also "linked list" would be just "list". It would be difficult to come up with better names since there would be name clashes. -- /Jacob Carlborg
Jun 10 2015
next sibling parent "Andrea Fontana" <nospam example.com> writes:
On Wednesday, 10 June 2015 at 07:50:24 UTC, Jacob Carlborg wrote:
 On 2015-06-09 20:49, Brad Anderson wrote:

 1.  vector/array
 2.  hash map
 3.  hash set
 4.  flat map
 5.  flat set
 6.  map
 7.  set
 8.  deque
 9.  stack
 10. queue
 11. linked list
 12. hash multimap
 13. hash multiset
 14. flat multimap
 15. flat multiset
 16. multimap
 17. multiset
 18. priority queue
 I think RedBlackTree isn't a very good name though. It's long 
 and is an
 implementation detail.
In that case you would need to remove "hash" and perhaps "flat" from all containers to be consistent. Also "linked list" would be just "list". It would be difficult to come up with better names since there would be name clashes.
It would be useful to have trees, heaps, (un)directed graphs...
Jun 10 2015
prev sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Wednesday, 10 June 2015 at 07:50:24 UTC, Jacob Carlborg wrote:
 On 2015-06-09 20:49, Brad Anderson wrote:

 1.  vector/array
 2.  hash map
 3.  hash set
 4.  flat map
 5.  flat set
 6.  map
 7.  set
 8.  deque
 9.  stack
 10. queue
 11. linked list
 12. hash multimap
 13. hash multiset
 14. flat multimap
 15. flat multiset
 16. multimap
 17. multiset
 18. priority queue
 I think RedBlackTree isn't a very good name though. It's long 
 and is an
 implementation detail.
In that case you would need to remove "hash" and perhaps "flat" from all containers to be consistent. Also "linked list" would be just "list". It would be difficult to come up with better names since there would be name clashes.
I wasn't suggesting those as the names. I just used "hash" because it's familiar to everyone (whereas C++'s unordered_map is a somewhat unusual name).
Jun 10 2015
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/09/2015 08:49 PM, Brad Anderson wrote:
 Containers that should be in the standard library in order of preference
 (from my C++ STL and Boost experience primarily):

 1.  vector/array
 2.  hash map
 3.  hash set
 4.  flat map
 5.  flat set
 6.  map
 7.  set
 8.  deque
 9.  stack
 10. queue
 11. linked list
 12. hash multimap
 13. hash multiset
 14. flat multimap
 15. flat multiset
 16. multimap
 17. multiset
 18. priority queue
Also: persistent versions.
Jun 10 2015
prev sibling next sibling parent Justin Whear <justin economicmodeling.com> writes:
On Tue, 09 Jun 2015 10:05:24 -0700, Andrei Alexandrescu wrote:

 One would be a good pass of std.container, in particular (a) a design
 review with the DbI glasses on; (b) better documentation - sadly it
 seems to me so inadequate as to make containers themselves unusable; (c)
 investigate use of UFCS - std.container's design predates UFCS yet is a
 perfect fit for it, and most likely other cool language improvements
 we've added since.
 
 The other would be database connectivity. Erik Smith has shown some cool
 ideas at DConf, and I encourage him to continue working on them, but it
 seems to me this is an area where more angles mean more connectivity
 options.
My vote would be for containers. At EMSI we do make heavy use of MySQL but have our own idiomatic D wrapper around the C library and would have little incentive to switch to a generic ODBC implementation.
Jun 09 2015
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 Please help me choose what to work on next.
 One would be a good pass of std.container, in particular (a) a 
 design review with the DbI glasses on; (b) better documentation 
 - sadly it seems to me so inadequate as to make containers 
 themselves unusable; (c) investigate use of UFCS - 
 std.container's design predates UFCS yet is a perfect fit for 
 it, and most likely other cool language improvements we've 
 added since.
No plans to use std.allocator? I think containers are the next logical step after allocators, and will also serve as a proving ground for the allocator API. One thing I'd like to note is that for any database connectivity solution to be future-proof, it needs to be based on (or compatible with) async I/O. Seeing as we still don't have async I/O in Phobos, I don't know how much that would influence the implementation. There are some database drivers in the Vibe.d ecosystem, but my knowledge of these extends to just the above fact.
Jun 09 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/9/15 4:40 PM, Vladimir Panteleev wrote:
 On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu wrote:
 Please help me choose what to work on next.
 One would be a good pass of std.container, in particular (a) a design
 review with the DbI glasses on; (b) better documentation - sadly it
 seems to me so inadequate as to make containers themselves unusable;
 (c) investigate use of UFCS - std.container's design predates UFCS yet
 is a perfect fit for it, and most likely other cool language
 improvements we've added since.
No plans to use std.allocator? I think containers are the next logical step after allocators, and will also serve as a proving ground for the allocator API.
I agree. In fact RedBlackTree from dcollections used an allocator, and the apparatus is still pretty much in std.container.rbtree. Part of me hoped to make another stab at getting dcollections suitable for inclusion in Phobos (much of my design philosophy has changed over the last few years), but I don't think I have the cycles :( -Steve
Jun 09 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/9/15 1:56 PM, Steven Schveighoffer wrote:
 On 6/9/15 4:40 PM, Vladimir Panteleev wrote:
 On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu wrote:
 Please help me choose what to work on next.
 One would be a good pass of std.container, in particular (a) a design
 review with the DbI glasses on; (b) better documentation - sadly it
 seems to me so inadequate as to make containers themselves unusable;
 (c) investigate use of UFCS - std.container's design predates UFCS yet
 is a perfect fit for it, and most likely other cool language
 improvements we've added since.
No plans to use std.allocator? I think containers are the next logical step after allocators, and will also serve as a proving ground for the allocator API.
I agree. In fact RedBlackTree from dcollections used an allocator, and the apparatus is still pretty much in std.container.rbtree.
Interesting, I didn't know about that.
 Part of me hoped to make another stab at getting dcollections suitable
 for inclusion in Phobos (much of my design philosophy has changed over
 the last few years), but I don't think I have the cycles :(
Regarding projects that we discussed you are considering, I suggest we focus on putting std.container in good shape and opt for a redesign only if there are great benefits. Also, I think we should stay with libc-based I/O. Andrei
Jun 09 2015
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 9 June 2015 at 21:53:55 UTC, Andrei Alexandrescu 
wrote:
 Regarding projects that we discussed you are considering, I 
 suggest we focus on putting std.container in good shape and opt 
 for a redesign only if there are great benefits. Also, I think 
 we should stay with libc-based I/O.
Honestly, I think that std.container's API is a very good start. The main problems lie with ranges. We need to make sure that all of the range-based functions are consistent (e.g. accepting Take!R, TakeOne!R, TakeExactly!R, etc. where R is the range type for that container - we do that to some extent, but we're not consistent about it), and we need to add more functions that don't use ranges - in particular, remove functions which remove keys or values without having to use a range to do it. Honestly, while it's flexible, it's not exactly the finest hour for ranges when you have to do something like container.remove(container[].find(value).takeOne()); just to remove a value for the container. Most folks just want to do something like container.removeFirst(value); And containers are really the main place, I think, where iterators actually do better than ranges. So, while I definitely want the containers to continue to support ranges, I think that we should do a better job of having alternative functions which don't involve ranges if they don't need to for many of the basic cases. Then the simple cases won't have to use ranges just to do stuff like remove elements, and generic code and more complicated cases can use ranges as appropriate. But in general, while I do think that some polishing is needed for the std.container API, I also think that it's a very solid start. - Jonathan M Davis
Jun 09 2015
next sibling parent Miles Stoudenmire via Digitalmars-d <digitalmars-d puremagic.com> writes:
A container similar to folly's small_vector would be great to have (
https://github.com/facebook/folly/blob/master/folly/docs/small_vector.md)

On 9 June 2015 at 18:11, Jonathan M Davis via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 9 June 2015 at 21:53:55 UTC, Andrei Alexandrescu wrote:

 Regarding projects that we discussed you are considering, I suggest we
 focus on putting std.container in good shape and opt for a redesign only if
 there are great benefits. Also, I think we should stay with libc-based I/O.
Honestly, I think that std.container's API is a very good start. The main problems lie with ranges. We need to make sure that all of the range-based functions are consistent (e.g. accepting Take!R, TakeOne!R, TakeExactly!R, etc. where R is the range type for that container - we do that to some extent, but we're not consistent about it), and we need to add more functions that don't use ranges - in particular, remove functions which remove keys or values without having to use a range to do it. Honestly, while it's flexible, it's not exactly the finest hour for ranges when you have to do something like container.remove(container[].find(value).takeOne()); just to remove a value for the container. Most folks just want to do something like container.removeFirst(value); And containers are really the main place, I think, where iterators actually do better than ranges. So, while I definitely want the containers to continue to support ranges, I think that we should do a better job of having alternative functions which don't involve ranges if they don't need to for many of the basic cases. Then the simple cases won't have to use ranges just to do stuff like remove elements, and generic code and more complicated cases can use ranges as appropriate. But in general, while I do think that some polishing is needed for the std.container API, I also think that it's a very solid start. - Jonathan M Davis
-- -=Miles Stoudenmire=- miles.stoudenmire gmail.com emiles pitp.ca http://itensor.org/miles/
Jun 09 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/9/15 3:11 PM, Jonathan M Davis wrote:
 And containers are really the main place, I think, where iterators
 actually do better than ranges.
I think iterators would need to add huge value to warrant addition. -- Andrei
Jun 09 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 9 June 2015 at 22:37:32 UTC, Andrei Alexandrescu 
wrote:
 On 6/9/15 3:11 PM, Jonathan M Davis wrote:
 And containers are really the main place, I think, where 
 iterators
 actually do better than ranges.
I think iterators would need to add huge value to warrant addition. -- Andrei
I don't disagree. I think that we should stick with ranges. I just think that this is one of the few areas where ranges are actually worse than iterators, whereas in most other areas, they're better. The result is that some of the idioms that are used with iterators in C++ become increasingly unwieldy with ranges - finding an element and removing it from the container is probably the simplest example. So, I think that the main change that we need to make in that regard to std.container is to have more functions which operate based on keys or values passed to the function rather than just having range-based functions. So, we'd have functions like removeFirst, removeLast, removeAll, etc. which take values rather than having all of the container functions be range-based. Iterators wouldn't need to enter into it at all. - Jonathan M Davis
Jun 09 2015
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/9/15 7:38 PM, Jonathan M Davis wrote:
 On Tuesday, 9 June 2015 at 22:37:32 UTC, Andrei Alexandrescu wrote:
 On 6/9/15 3:11 PM, Jonathan M Davis wrote:
 And containers are really the main place, I think, where iterators
 actually do better than ranges.
I think iterators would need to add huge value to warrant addition. -- Andrei
I don't disagree. I think that we should stick with ranges. I just think that this is one of the few areas where ranges are actually worse than iterators, whereas in most other areas, they're better. The result is that some of the idioms that are used with iterators in C++ become increasingly unwieldy with ranges - finding an element and removing it from the container is probably the simplest example.
What's needed is a simple way to refer to exactly one element. I solved that problem in dcollections. -Steve
Jun 10 2015
prev sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 9 June 2015 at 21:53:55 UTC, Andrei Alexandrescu 
wrote:
 Also, I think we should stay with libc-based I/O.
Um, won't this mean that the library then won't be usable by any application that will want to do more than a few simultaneous queries? Database engines usually concentrate on concurrency, not latency, so non-asynchronous I/O is only adequate for simple programs.
Jun 09 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/9/15 6:20 PM, Vladimir Panteleev wrote:
 On Tuesday, 9 June 2015 at 21:53:55 UTC, Andrei Alexandrescu wrote:
 Also, I think we should stay with libc-based I/O.
Um, won't this mean that the library then won't be usable by any application that will want to do more than a few simultaneous queries? Database engines usually concentrate on concurrency, not latency, so non-asynchronous I/O is only adequate for simple programs.
He hasn't seen the contrast yet :) In any case, I have zero plans to abandon what I've been working on (glacially) over the last 3 years. -Steve
Jun 10 2015
parent "w0rp" <devw0rp gmail.com> writes:
Containers seems like the best thing to work on after the 
allocators. Then you could start using the allocators for the 
containers, so you can choose different allocation strategies for 
them.

I'd definitely like to see a standard container for sets, instead 
of the void[0][T] thing I usually do.
Jun 10 2015
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 Please help me choose what to work on next.
Well, containers matter for almost every program, whereas databases don't. They matter for a lot of programs, but many programs don't have them. And we already have std.container, even if it needs more work. So, I think that it would make more sense if we std.container were completed than to emark on std.database. And we've been saying for years now (be it right or wrong) that one of the main reasons that we weren't doing more with std.container is that we were expecting to have to change it after std.allocator was done. So, it seems well past time that std.container gets sorted out. Of course, that doesn't necessarily mean that you need to be the one who does it, but you are the one who started it, so it would make sense if you tackled it next. - Jonathan M Davis
Jun 09 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 9 June 2015 at 21:20:14 UTC, Jonathan M Davis wrote:
 So, I think that it would make more sense if we std.container 
 were completed than to embark on std.database.
I should probably add that since we already have std.container, doing containers next is a matter of polishing it and finishing it; it doesn't necessarily need a lot of bake time at this point. However, I expect that if we want a top-notch database implementation, it's going to need some bake time. So, while it might make sense to start bouncing around ideas on a database API to get that process started, it would make a lot more sense to focus on std.container first. - Jonathan M Davis
Jun 09 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/9/15 3:17 PM, Jonathan M Davis wrote:
 On Tuesday, 9 June 2015 at 21:20:14 UTC, Jonathan M Davis wrote:
 So, I think that it would make more sense if we std.container were
 completed than to embark on std.database.
I should probably add that since we already have std.container, doing containers next is a matter of polishing it and finishing it; it doesn't necessarily need a lot of bake time at this point. However, I expect that if we want a top-notch database implementation, it's going to need some bake time. So, while it might make sense to start bouncing around ideas on a database API to get that process started, it would make a lot more sense to focus on std.container first. - Jonathan M Davis
Well what do you know. I got http://spottedtiger.tripod.com/D_Language/D_Support_Projects_XP.html going and have a branch with etc.c.odbc working like a charm. I'll submit that soon. With regard to which to work on, I've made my decision: http://38.media.tumblr.com/c51baf89e2f85065d669f3080c2b5cdb/tumblr_inline_n3rsrwCCcQ1qj3ir1.jpg Andrei
Jun 09 2015
parent "Joakim" <dlang joakim.fea.st> writes:
On Tuesday, 9 June 2015 at 22:42:15 UTC, Andrei Alexandrescu 
wrote:
 With regard to which to work on, I've made my decision: 
 http://38.media.tumblr.com/c51baf89e2f85065d669f3080c2b5cdb/tumblr_inline_n3rsrwCCcQ1qj3ir1.jpg
For those voting, you can stop now: he made his decision. The Spanish in the image above translates to "Why not both?"
Jun 10 2015
prev sibling next sibling parent "ponce" <contact gam3sfrommars.fr> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 Please help me choose what to work on next.
Containers!
Jun 09 2015
prev sibling next sibling parent "weaselcat" <weaselcat gmail.com> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 ...
I think the overall benefit would be from containers, as everyone uses containers.
Jun 09 2015
prev sibling next sibling parent reply "ZombineDev" <valid_email he.re> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 Please help me choose what to work on next.


 Andrei
As others have said - definitely containers! A lot more applications use them. There is too much differences between databases (SQL and No-SQL) to cover them effectively under a single interface. Every language has at least a couple of libraries that take very different approaches with different levels of abstraction and so on. There is no single correct design. So for now I think it it's better to leave databases to dub. On the other hand containers is the right place to do an opinionated design which integrates well with other parts of Phobos (std.algorithm, std.range, std.ex.allocator and maybe some policy-based design for logging backed by std.ex.logger). See Brad Anderson's comment other for more specific action items. One other important thing that needs to be addressed is library-based AA. Igor Stepanov and Martin Nowak (and others) have been trying to solve this for a long time but there some issues that may require breaking changes. This area requires more serious attention because there is no easy solution. Also currently dynamic arrays, exceptions and other language features are also heavily tied to the GC, preventing them to be used in projects for which GC is not appropriate. After all the good work that has gone in std.allocator, it's a shame if two of D's most used built-in types can't benefit. The more we can untie the implementation of dynamic arrays from dmd and druntime, the faster we can deliver improvements in those areas. There were some nice ideas here[1] which I really hope to be realized. So any ideas on how to move more the implementation of AAs and dynamic arrays to user-land (instead of compiler-land), so one could easily decide on the memory management strategy and data-structure that back the AA, without replacing all usages of the built-in AA with a library type from std.container? It may be more about implementation than language changes, but recently I was thinking about what a language enhanced may look like: { // Swap the implementation of built-in AAs //and use HashMap instead in this scope alias __aa(K,V) = std.container.hm.HashMap!(K, V); ... } Or: // Templatize a module by memory management strategy (and more). // Like specifying versions on the command-line, but // as powerful as templates: enum desired_mms = MMStrategy.refCounting; import image_processing!desired_mms : Image; // Image is a refcounted or GC class, depending on // the module template parameter [1]: http://beta.forum.dlang.org/thread/m0bdgg$1t7j$1 digitalmars.com
Jun 09 2015
next sibling parent "ZombineDev" <valid_email he.re> writes:
On Wednesday, 10 June 2015 at 00:26:23 UTC, ZombineDev wrote:
 After all the good work that has gone in std.allocator, it's a 
 shame if two of D's most used built-in types can't benefit. The 
 more we can untie the implementation of dynamic arrays from dmd 
 and druntime, the faster we can deliver improvements in those 
 areas.
Here I meant mostly Associative arrays and perhaps dynamic array.
Jun 09 2015
prev sibling parent "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
On Wednesday, 10 June 2015 at 00:26:23 UTC, ZombineDev wrote:

 // Templatize a module by memory management strategy (and more).
 // Like specifying versions on the command-line, but
 // as powerful as templates:

 enum desired_mms = MMStrategy.refCounting;
 import image_processing!desired_mms : Image;
 // Image is a refcounted or GC class, depending on
 // the module template parameter

 [1]: 
 http://beta.forum.dlang.org/thread/m0bdgg$1t7j$1 digitalmars.com
Why not something like alias HashMap (T) = std.container.HashMap!(T, MMStrategy.refCounting) which is available now, just depends on how the template arts for a given container are set up.
Jun 09 2015
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Well I got this working. Please review:

https://github.com/D-Programming-Language/phobos/pull/3398


Andrei
Jun 09 2015
prev sibling parent reply "Andrea Fontana" <nospam example.com> writes:
On Tuesday, 9 June 2015 at 17:05:19 UTC, Andrei Alexandrescu 
wrote:
 Please help me choose what to work on next.


 Andrei
My vote goes for std.containers! Anyway imho std.database should be a generic database module. Not a sql-oriented one. ODBC is sql-oriented! If not, call it std.sql or std.database.sql. Andrea
Jun 10 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/10/15 12:36 AM, Andrea Fontana wrote:
 ODBC is sql-oriented! If not, call it std.sql or std.database.sql.
Well it's etc.c.odbc :o). -- Andrei
Jun 10 2015