www.digitalmars.com         C & C++   DMDScript  

D - STL as part of the D language

reply FlyTox <rox271 hotmail.com> writes:
I just discovered the D language a couple of days ago and it looks exciting.

Having programmed for quite a long time on C++, there is one thing that 
I really like and would really appreciate to see included in a new 
language : STLs. Today I cannot think without STLs, though the syntax is 
terrible. But the benefit is huge. I can see that D is digging into 
dynamic arrays and even maps. This still remains a bit basic compared 
with STL. Any chance to have more collections (sets, generic maps, 
queues etc...), iterators, comparison operators or the idea is to keep 
STL in a external template lib?
Apr 25 2004
next sibling parent reply Stephan Wienczny <wienczny web.de> writes:
FlyTox wrote:
 I just discovered the D language a couple of days ago and it looks 
 exciting.
 
 Having programmed for quite a long time on C++, there is one thing that 
 I really like and would really appreciate to see included in a new 
 language : STLs. Today I cannot think without STLs, though the syntax is 
 terrible. But the benefit is huge. I can see that D is digging into 
 dynamic arrays and even maps. This still remains a bit basic compared 
 with STL. Any chance to have more collections (sets, generic maps, 
 queues etc...), iterators, comparison operators or the idea is to keep 
 STL in a external template lib?
 

Matthew is writing a DTL (D Template Library). I think he will do a good job as he is experienced in writing template stuff for Stlsoft. DTL will be available in one of the next dmd releases. Stephan
Apr 25 2004
parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
"Stephan Wienczny" <wienczny web.de> wrote in message
news:c6g3m0$18je$1 digitaldaemon.com...
 FlyTox wrote:
 I just discovered the D language a couple of days ago and it looks
 exciting.

 Having programmed for quite a long time on C++, there is one thing that
 I really like and would really appreciate to see included in a new
 language : STLs. Today I cannot think without STLs, though the syntax is
 terrible. But the benefit is huge. I can see that D is digging into
 dynamic arrays and even maps. This still remains a bit basic compared
 with STL. Any chance to have more collections (sets, generic maps,
 queues etc...), iterators, comparison operators or the idea is to keep
 STL in a external template lib?

Matthew is writing a DTL (D Template Library). I think he will do a good job as he is experienced in writing template stuff for Stlsoft. DTL will be available in one of the next dmd releases.

Thanks. Now I've got to live up to your expectation! :)
Apr 25 2004
parent reply Ant <duitoolkit yahoo.ca> writes:
On Sun, 25 Apr 2004 23:38:46 +1000, Matthew wrote:

 
 
 Thanks. Now I've got to live up to your expectation! :)

I don't get why this is not an open project. can you explain? My imagination is imagining the worst possibilities. Thanks. Ant
Apr 25 2004
parent "Matthew" <matthew.hat stlsoft.dot.org> writes:
"Ant" <duitoolkit yahoo.ca> wrote in message
news:pan.2004.04.25.15.57.58.568212 yahoo.ca...
 On Sun, 25 Apr 2004 23:38:46 +1000, Matthew wrote:

 Thanks. Now I've got to live up to your expectation! :)

I don't get why this is not an open project. can you explain?

Because I chose to do it this way. When a "DTL" was mooted several months ago, there was no clear idea as to how a container/algorithm library would work in D; the only attempts either used "classic" (i.e. pre-STL) style containers, or aped STL directly. The idea floated at the time was that people were going to work on different mechanisms, and we would eventually compare the different implementations, and take the best ideas from them all. I've been working on one that, rather than prescribes a single enumeration model, supports a diversity of them with maximal efficiency. AFAIK, no-one else has worked on a DTL. I've kept people up to speed with what's intended, but there can hardly be much active collaboration, because: (i) until about two days ago it was still in a state of massive flux conceptually (ii) the compiler does not yet support the majority of the intended implementation, so how could we collaborate sensibly.
 My imagination is imagining the worst possibilities.

Thanks! You're *more* than welcome to go with the initial plan, and write something yourself, and we can compare them when they're both ready and extract the best.
Apr 25 2004
prev sibling next sibling parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
FlyTox wrote:

 I just discovered the D language a couple of days ago and it looks 
 exciting.

 Having programmed for quite a long time on C++, there is one thing 
 that I really like and would really appreciate to see included in a 
 new language : STLs. Today I cannot think without STLs, though the 
 syntax is terrible. But the benefit is huge. I can see that D is 
 digging into dynamic arrays and even maps. This still remains a bit 
 basic compared with STL. Any chance to have more collections (sets, 
 generic maps, queues etc...), iterators, comparison operators or the 
 idea is to keep STL in a external template lib?

DTL is in early development by Matthew. We should all be able to contribute soon. -- -Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
parent "Matthew" <matthew.hat stlsoft.dot.org> writes:
"J Anderson" <REMOVEanderson badmama.com.au> wrote in message
news:c6g3ra$18h9$2 digitaldaemon.com...
 FlyTox wrote:

 I just discovered the D language a couple of days ago and it looks
 exciting.

 Having programmed for quite a long time on C++, there is one thing
 that I really like and would really appreciate to see included in a
 new language : STLs. Today I cannot think without STLs, though the
 syntax is terrible. But the benefit is huge. I can see that D is
 digging into dynamic arrays and even maps. This still remains a bit
 basic compared with STL. Any chance to have more collections (sets,
 generic maps, queues etc...), iterators, comparison operators or the
 idea is to keep STL in a external template lib?

DTL is in early development by Matthew. We should all be able to contribute soon.

Very soon, I hope. Just waiting on some serious language/compiler enhancements by our inexplicably silent <G> leader. (Sorry Walter, but you *did* say you wanted D to knock the pants of everything, and C++ is very good at templates ...)
Apr 25 2004
prev sibling next sibling parent reply FlyTox <rox271 hotmail.com> writes:
Thanks for the information. So this means that DTL will be an external 
lib. Actually I was wondering why not have STL as part of the 
programming language. Who is nowaday doing any serious OOP without 
advanced object collections?

Also most COM libraries have Variant interface arrays with iterators and 
since D is offering an easy way to implement COM interfaces that would 
be nice to have collections as part of the language. Will DTL allow an 
easy COM collection implementation in order to interface with VB apps?

Should we have collections in an external lib? I believe that the next 
generation of programming language cannot ignore collections which are a 
major aspect of the nowaday OOP.
Apr 25 2004
next sibling parent J Anderson <REMOVEanderson badmama.com.au> writes:
FlyTox wrote:

 Thanks for the information. So this means that DTL will be an external 
 lib.

DTL will come with D like STL comes with C++. -- -Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
prev sibling next sibling parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
What is wrong with a standard library? Of course the DTL or something like
it should be standardized and included into Phobos. (I have little doubt,
it will.)

The language itself should rather stay as slim as possible, just giving the
flexibility and power to define everything in the library.

That is just the core concept of most modern programming languages: to have
the language spec slim and general to allow powerful libraries with high
performance and clean syntax.

FlyTox wrote:

 Thanks for the information. So this means that DTL will be an external
 lib. Actually I was wondering why not have STL as part of the
 programming language. Who is nowaday doing any serious OOP without
 advanced object collections?
 
 Also most COM libraries have Variant interface arrays with iterators and
 since D is offering an easy way to implement COM interfaces that would
 be nice to have collections as part of the language. Will DTL allow an
 easy COM collection implementation in order to interface with VB apps?
 
 Should we have collections in an external lib? I believe that the next
 generation of programming language cannot ignore collections which are a
 major aspect of the nowaday OOP.

Apr 25 2004
next sibling parent "Matthew" <matthew.hat stlsoft.dot.org> writes:
 That is just the core concept of most modern programming languages: to have
 the language spec slim and general to allow powerful libraries with high
 performance and clean syntax.

Exactly right! (Now, why couldn't I say that as simply ... <g>)
 FlyTox wrote:

 Thanks for the information. So this means that DTL will be an external
 lib. Actually I was wondering why not have STL as part of the
 programming language. Who is nowaday doing any serious OOP without
 advanced object collections?

 Also most COM libraries have Variant interface arrays with iterators and
 since D is offering an easy way to implement COM interfaces that would
 be nice to have collections as part of the language. Will DTL allow an
 easy COM collection implementation in order to interface with VB apps?

 Should we have collections in an external lib? I believe that the next
 generation of programming language cannot ignore collections which are a
 major aspect of the nowaday OOP.


Apr 25 2004
prev sibling parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
Norbert Nemec wrote:

What is wrong with a standard library? 

that overlap STL. We want people to use the D way, not STL's. -- -Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
parent Norbert Nemec <Norbert.Nemec gmx.de> writes:
J Anderson wrote:

 Norbert Nemec wrote:
 
What is wrong with a standard library?

that overlap STL. We want people to use the D way, not STL's.

I never talked of the C++ STL - I talked of *a* standard library. Of course, ideas will be picked from the C++ STL, but we can certainly do a better job that *that* in D. And how it is called in the end - who knows. Probably just "part of the D library". (If it really is standard, it does not need a name claiming to be standard...)
Apr 25 2004
prev sibling next sibling parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
"FlyTox" <rox271 hotmail.com> wrote in message
news:c6g7lv$1eqm$1 digitaldaemon.com...
 Thanks for the information. So this means that DTL will be an external
 lib.

Yes.
 Actually I was wondering why not have STL as part of the
 programming language.

Two reasons. First, language vs library is a grey area, and we choose library in this case. Second, it is way too complex to have it all in the language. Third, it is designed to be extensible just as STL was (but rarely is; ahem ... http://stlsoft.org/), which leaves me, at least, guessing how it could be a language thing.
 Who is nowaday doing any serious OOP without
 advanced object collections?

<opinionated rant>Who is doing any serious OOP? It's a dead duck!</opinionated rant> Who is doing any serious programming these days without serialisation (saving / loading), serialisation (synchronisation), XML, IO, etc. etc. etc. Are all these things to be forced into the language? Obviously not. And in answer to your question, I know of at least one really smart chap who rarely, if ever, uses STL. _Even_ me, a big big fan of STL, eschews it often. It's all horses for courses, in my opinion. In a slightly unrelated note, Walter and I have a continuing debate over my propensity for providing free functions along with class(es) when writing D libraries. He believes there should be only the class-based approach, but I see D as a broad church, which should welcome programmers with as diverse skills as is practicable. For example, we're just arguing over the fine details of my ExeModule module, which provides an ExeModule class that throws exceptions, and ExeModule_Load(), ExeModule_AddRef(), ExeModule_GetSymbol(), etc functions, which afford a more C like approach. In the same vein, I intend to support many different enumeration models for DTL, so long as they do not cause a sacrifice in efficiency, ease-of-use, robustness or (within reason) maintainability.
 Also most COM libraries have Variant interface arrays with iterators and
 since D is offering an easy way to implement COM interfaces that would
 be nice to have collections as part of the language. Will DTL allow an
 easy COM collection implementation in order to interface with VB apps?

It will, via extensions, in just the same way that COMSTL, UNIXSTL, WinSTL, InetSTL (just alpha'd on the STLSoft newsgroup today; news://news.digitalmars.com/c++.stlsoft) all provide STL-like functionality over disparate technology-specific and operating system APIs.
 Should we have collections in an external lib? I believe that the next
 generation of programming language cannot ignore collections which are a
 major aspect of the nowaday OOP.

Yes, for the practical reasons I've given above.
Apr 25 2004
parent Ant <duitoolkit yahoo.ca> writes:
On Sun, 25 Apr 2004 23:48:51 +1000, Matthew wrote:

 
 It's all horses for courses, in my opinion. In a slightly unrelated note,
Walter
 and I have a continuing debate over my propensity for providing free functions
 along with class(es) when writing D libraries. He believes there should be only
 the class-based approach, but I see D as a broad church, which should welcome
 programmers with as diverse skills as is practicable.

Walter is right. Ant
Apr 25 2004
prev sibling parent Ilya Minkov <minkov cs.tum.edu> writes:
FlyTox schrieb:

 Should we have collections in an external lib? I believe that the next 
 generation of programming language cannot ignore collections which are a 
 major aspect of the nowaday OOP.

Very true. But nontheless: you cannot put everything into the core language, something will be still missing. Libraries are a sort of extension mecanism, which can fill this gap. So why not be everything a library? Well, it works in some languages, but with a C descentant this causes a bit more problems. Besides, putting everything into a compiler is hard and probably somewhat error-prone. So a balance has to be found. The core language in D already gives a lot compared to most languages (linear and associative container - the ones needed most often), and gives a solid basis for everything else... I think it should stay this way so far. -eye
Apr 25 2004
prev sibling parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
"FlyTox" <rox271 hotmail.com> wrote in message
news:c6g241$16c1$1 digitaldaemon.com...
 I just discovered the D language a couple of days ago and it looks exciting.

 Having programmed for quite a long time on C++, there is one thing that
 I really like and would really appreciate to see included in a new
 language : STLs. Today I cannot think without STLs, though the syntax is
 terrible. But the benefit is huge. I can see that D is digging into
 dynamic arrays and even maps. This still remains a bit basic compared
 with STL. Any chance to have more collections (sets, generic maps,
 queues etc...), iterators, comparison operators or the idea is to keep
 STL in a external template lib?

Here's the basic idea. DTL collections will support compile-time polymorphism by default, and runtime polymorphism as a parameterisable option. To my knowledge this has not been successfully done in C++ - although I have the beginnings of a VirtualSTL (another STLSoft thing) which I'll mature throughout this year (it's going to feature in my next book on STL) - and will represent a massive boon to the D user community, because fans of both the STL and the .NET/Java's interface-based approaches will be equally served. The enumeration models supported are (or will be, just as soon as poor Walter's satisfied my every whim; whip crack away!): 1. Enum#1 - foreach All DTL containers will present opApply() and will be freachable 2. Enum#2 - iterators All DTL sequence containers - others are undecided at present - will present an iterator based representation to their managed sequence, in the form of begin() and end(). Because of differences in D's support for operator overloading, however, (see point B. below) there are differences in how iterators must be used. But this will be syntactic only. FYI, the current iterator for the List template, ListIterator, presents the following interface: void next(); value_type value(); // Can be accessed as property, as in "X x = it.value;" int opEquals(); Since *all* DTL containers will have iterators that have the same interface, there's no confusion with array (op*) syntax - the point simply does not exist. Circular argument, to be sure, but it works nonetheless. 3. Enum#3 - Ranges This is a concept being developed by myself and John Torjo, and features in my soon-to-be-published book "Imperfect C++", as well as featuring even more in the next book on STL. We're writing a pair of articles on the subject for CUJ later in the year. At present, Ranges exists in a small STLSoft library (not yet released) and a reasonably complete Boost library (including filters, John's speciality). Anyway, that's all very well, but what do you care? Well, the DTL Enum#3 model will use Ranges to present freachable results from methods similar to the Ruby Enumerable module (see http://ruby-lang.org; download the Win32 version, as there's a free electronic copy of The Pragmatic Programmers book on Ruby!). At the moment it is my hope that the Enum#3 methods will be provided, by default, by a Ranges mixin, and Walter is currently working out whether my proposed mixin protocol - which is dirt simple, just pop the things in the mixing class unless that class already has one of the exact signature - is feasible, and whether he can do it before he has a stress episode. ;) To be less waffly and more informative, the Enum#3 model will support the following methods (some of which may not be instantiable, of course, based on container value type): template(F) range_type collect(F f); boolean contains(value_type comperand); template(F) boolean detect(F f); template(F) boolean detect(F f, out value_type value); value_type[] entries(); value_type min(); template(F) value_type min(F f); value_type max(); template(F) value_type max(F f); template(F) range_type reject(F f); template(F) range_type select(F f); range_type sort(); template(F) range_type sort(F f); 4. Enum#4 - an interface of your choice All of Enum#1-3 are compile-time typed. However, with some clever use of default template parameters (already added for next release), mixins (to be determined) and specialisation (extant), you will be able to parameterise your container template on an interface (for which you've provided the appropriate mixins / adaptors; that bit's still unclear, as I've not been able to play with actual compiled code yet) and your container will, in addition to all its normal behaviour, derive from and fully implement (semantically, I mean) that interface. This means that (i) you can live in Java/.NET Object-based collection world, if you wish, and (ii) you can write write-once non-template functions for a given interface, and use it with arbitrary DTL container types which you have parameterised to your interface. That all sounds good - at least I hope it does - but there are some restrictions imposed by D on us, so there will be some changes to the way people use collections, especially to those (of us) who love STL: A. There is no implicit template instantiation. This means that the use of free function algorithms is not going to be useful. Most of the things anyone would want to do with a collection of elements will be covered by foreach (Enum#1) & Ranges (Enum#3), but it is of course inevitable that there will be special cases. In such cases, explicit instantiation will have to suffice. But given the fact that *all* DTL collections will have a standard, sensible and mandatory set of member types, and because D has a typeof operator, I think the pain will be slight, if it's even noticeable. B. There is no user-definable operator *() - I called it opDeref when I pleaded in vain with Walter for it. (Actually, I am being dramatic for the crowd, my plea was a half-enthused "do we have it? Is it going to exist?" while keeping my fingers crossed that he'd say no, which he did. With the restrictions imposed by the _lack_ of op*(), we will have less lee-way to pansy around C compatibility (arrays) and all collection manipulation will be much simpler and consistent.) The ramifications of this is that there will be no compatibility in user-defined collections with built-in arrays access by pointer. This is inline with with D preference for array slices over pointers. Once you give in/up and accept this, the world becomes altogether simpler. There will be a high degree of compatibility between DTL collections and built-in arrays, in the form of slicing, via Ranges (Enum#3). Hope that explains the current thrust of the development Cheers -- Matthew Wilson Contributing editor, C/C++ Users Journal (www.synesis.com.au/articles.html#columns) STLSoft moderator (http://www.stlsoft.org) "I can't sleep nights till I found out who hurled what ball through what apparatus" -- Dr Niles Crane -------------------------------------------------------------------------------
Apr 25 2004
next sibling parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
Matthew wrote:
<snip>

Thanks for that update Matthew.  I can't wait <g>/

-- 
-Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
Oh no, not more pressure!!

<G>

"J Anderson" <REMOVEanderson badmama.com.au> wrote in message
news:c6gksu$24po$1 digitaldaemon.com...
 Matthew wrote:
 <snip>

 Thanks for that update Matthew.  I can't wait <g>/

 -- 
 -Anderson: http://badmama.com.au/~anderson/

Apr 25 2004
next sibling parent reply Stephan Wienczny <wienczny web.de> writes:
What is the current state of DTL?

How finished are DTL and the compiler changes?

Matthew wrote:
 Oh no, not more pressure!!
 
 <G>
 
 "J Anderson" <REMOVEanderson badmama.com.au> wrote in message
 news:c6gksu$24po$1 digitaldaemon.com...
 
Matthew wrote:
<snip>

Thanks for that update Matthew.  I can't wait <g>/

-- 
-Anderson: http://badmama.com.au/~anderson/


Apr 25 2004
parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
 What is the current state of DTL?

Enum#1 - done for all currently implemented containers Enum#2 - done for Vector and List. No algorithms done yet. (In fact, I'm hoping others will pitch in on this, as soon as I can get it out there.) Enum#3 - waiting for support for member template functions and mixins Enum#4 - concept tested, but waiting for mixins Containers - Vector, List, Set, Queue, Stack, Map - all in various stages of Enum#N completeness. Once it's done for one, however, it'll be done for all within a day Ranges - IntRange
 How finished are DTL and the compiler changes?

AFAIK, Compiler changes: Default template parameters - done Member templates - Walter's on this now Mixins - Waiting DTL completeness: As above
 Matthew wrote:
 Oh no, not more pressure!!

 <G>

 "J Anderson" <REMOVEanderson badmama.com.au> wrote in message
 news:c6gksu$24po$1 digitaldaemon.com...

Matthew wrote:
<snip>

Thanks for that update Matthew.  I can't wait <g>/

-- 
-Anderson: http://badmama.com.au/~anderson/



Apr 25 2004
parent Stephan Wienczny <wienczny web.de> writes:
Good to hear that...

Matthew wrote:
What is the current state of DTL?


  Enum#1 - done for all currently implemented containers

  Enum#2 - done for Vector and List. No algorithms done yet. (In fact, I'm
hoping
 others will pitch in on this, as soon as I can get it out there.)

  Enum#3 - waiting for support for member template functions and mixins

  Enum#4 - concept tested, but waiting for mixins
 
  Containers - Vector, List, Set, Queue, Stack, Map - all in various stages of
 Enum#N completeness. Once it's done for one, however, it'll be done for all
 within a day
  Ranges - IntRange
 
 
How finished are DTL and the compiler changes?

AFAIK, Compiler changes: Default template parameters - done Member templates - Walter's on this now Mixins - Waiting DTL completeness: As above

Apr 25 2004
prev sibling parent J Anderson <REMOVEanderson badmama.com.au> writes:
Matthew wrote:

Oh no, not more pressure!!

<G>
  

Thanks for the effort Matthew. -- -Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
prev sibling parent reply Bastiaan Veelo <Bastiaan.N.Veelo ntnu.no> writes:
Matthew wrote:

(snipped a bit)
 The enumeration models supported are (or will be, just as soon as poor Walter's
 satisfied my every whim; whip crack away!):
 
 1. Enum#1 - foreach
  
 2. Enum#2 - iterators
 
 3. Enum#3 - Ranges
 
 4. Enum#4 - an interface of your choice

Apropos enumeration models, are you familiar with circulators? http://www.cgal.org/Manual/doc_html/support_lib/Handles_and_Circs/Chapter_main.html#Section_2 http://www.cgal.org/Manual/doc_html/use_of_stl/Chapter_Circulator_stl.html I think it would be nice to have support for circular data structures in the standard library. They have their uses in geometry. Regards, Bastiaan.
Apr 25 2004
parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
"Bastiaan Veelo" <Bastiaan.N.Veelo ntnu.no> wrote in message
news:c6gq4n$2dge$1 digitaldaemon.com...
 Matthew wrote:

 (snipped a bit)
 The enumeration models supported are (or will be, just as soon as poor


 satisfied my every whim; whip crack away!):

 1. Enum#1 - foreach

 2. Enum#2 - iterators

 3. Enum#3 - Ranges

 4. Enum#4 - an interface of your choice

Apropos enumeration models, are you familiar with circulators?

 http://www.cgal.org/Manual/doc_html/use_of_stl/Chapter_Circulator_stl.html

 I think it would be nice to have support for circular data structures in
 the standard library. They have their uses in geometry.

I'd certainly be interested in seeing how these might fit it. I would hope you'll volunteer some MDTL extension time when the compiler's ready and I've released 0.1. ;)
Apr 25 2004
parent Bastiaan Veelo <Bastiaan.N.Veelo ntnu.no> writes:
Matthew wrote:
 
 http://www.cgal.org/Manual/doc_html/support_lib/Handles_and_Circs/Chapter_main.html#Section_2
 
http://www.cgal.org/Manual/doc_html/use_of_stl/Chapter_Circulator_stl.html

I think it would be nice to have support for circular data structures in
the standard library. They have their uses in geometry.

I'd certainly be interested in seeing how these might fit it. I would hope you'll volunteer some MDTL extension time when the compiler's ready and I've released 0.1. ;)

I can see if I am capable of that, but it will have to wait until the fall. I have to deliver a thesis first --- me hanging out in this NG is like doing the ioning... ;) Bastiaan.
Apr 26 2004