www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D cannot support generic programming

reply "Matthew" <admin.hat stlsoft.dot.org> writes:
I'm almost at the end of my tether here. There are so many problems with the
language/compiler, that I'm wondering
seriously whether it's going to be possible to have a decent generic template
library.

Before I preface this, I should reiterate the "modes" of DTL, to inform on any
thoughts anyone might have on this:

1. foreach - client code uses foreach. This all works well for all containers.
The problem with this is only that it's
an unsophisticated view of looking at a container: you get everything, and have
to do all filtering and transformations
yourself in client code. This can result in verbose and tedious coding. Good
libraries should provide more power.

2. Transformations/filtering - based on "Ranges" (a concept John Torjo and I
concocted for C++, but which is eminently
suitable for D.). Basically, each container provides a set of methods
(eventually via a mixin) that returns a "range".
For example, the "select()" method takes either a function/delegate/functor
predicate, and returns a freachable range
that "contains" only the elements matching the predicate, e.g.

    List!(int)        list = . . .; // Assume it's filled with numbers 0 - 9
    bool IsOdd(int i)
    {
        return 0 != (i & 0x2);
    }
    foreach(int i; list.select(IsOdd))
    {
        printf("%d ", i);
    }

This will print out "1 3 5 7 9"

The power of this is that the returned "range" itself provides the same
transformation methods, which means that the
transformations are composable, as in:

    List!(int)        list = . . .; // Assume it's filled with numbers 0 - 9
    bool IsOdd(int i)
    {
        return 0 != (i & 0x2);
    }
    int Times2(int i)
    {
        return 2 * i;
    }
    foreach(int i; list.select(IsOdd).collect(Times2))
    {
        printf("%d ", i);
    }

This will print out "2 6 10 14 18"

I'm sure you can see the power in this approach, as well as the potential
efficiency savings, where the transformation
"criteria" may be tunnelled into the original container's foreach (I've not
done that yet).

3. Interface-based containers

Each container in DTL is declared as follows:

    template Vector(T, B = EmptyBase)
    {
        class Vector
            : public BaseSelector!(B).selected_type


This means that by default the container will be do as the STL does, and will
just be all compile-time typing, e.g.

    alias    Vector!(int)     int_vector_t;

    int_vector_t    v = new ...;

    v[10] = 3;
    size_t n = v.length;

    int    r = v[0] + v[1];

Such a type is great, as long as we want to manipulate it in/with code that
knows its exact type.

Alternatively, we can also support the old-style Java approach, whereby one can
specify a base interface, e.g.

    void dump(IObjectContainer c)
    {
        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
        {
            printf("%.*s ", e.nextElement().toString());
        }
        printf("\n");
    }

    alias    Vector!(int, IObjectContainer)    int_vector_IObjectContainer_t;
    alias    List!(int, IObjectContainer)        int_list_IObjectContainer_t;

    int_vector_IObjectContainer_t   v = new ...;
    int_list_IObjectContainer_t        l = new ...;

    dump(v);
    dump(l);

Now we can pass instances of any container implementing IObjectContainer this
to *any* function that knows how to
manipulate that interface.

[btw: both of the above forms work now and, fwiw, I hope to release some of the
containers supporting this in the next
c/o days.]

The downside to the IObjectContainer type approach is that (i) fundamental
types must be "boxed" (I've already worked up
std.box), and (ii) it's not generic, since one must downcast object types to
their "real" types from Object. Pretty
pukey stuff, except in the minority of circumstances where the IObject-stuff is
what you're after (e.g. a message
board).

Further to these two approaches, which I've been working on today, is
inheritance via parameterised-interfaces, e.g.

    template dump(T) { void dump(IContainer!(T) c)
    {
        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
        {
            T    t    =    e.nextElement();

            // ... write generically, presumably via writef() (which I've not
used yet <g>)
        }
        printf("\n");
    }

    alias    IContainer!(int)                         int_IContainer_t;
    alias    Vector!(int, int_IContainer_t)    int_vector_IContainer_int_t;
    alias    List!(int, int_IContainer_t)        int_list_IContainer_int_t;

    int_vector_IContainer_int_t      v = new ...;
    int_list_IContainer_int_t            l = new ...;

    dump(v);
    dump(l);

Note: this approach does _not_ require boxing or downcasting.

Alas, this crashes the compiler. Sigh ...

4. Iterator based approach. There are two reasons why STL-iterators are not
possible in D: (i) there is no implicit
instantiation, and (ii) there is no operator *(). Neither of these preclude the
creation of iterators - and I've done a
little of that - including adaptors for arrays/pointers, but it means they're
not really very usable. The only way to
use them "generically" in algorithms would be polymorphically if the iterator
classes derived from interfaces, and that
would preclude efficiency mechanisms (such as Random Access iterator
advancement in STL), without some nasty hacks.

It's my belief that iterators are not for D, and that we thus have the modes
1-3 described above.

And now on to my litany of woes ...

Some tasters:

1. Beyond a significant, but indefinable level of complexity, the
linker/compiler loose the ability to find all required
symbols, while chopping out any *unrelated* parts makes them locatable again.
Boil it down? I wish!!

Nonetheless, that's a bug in the compiler/linker, and doesn't get my blood up,
since one can usually find workarounds,
and at this stage a little bugginess is not the crime of the century.

2. Templates in D are instantiated en masse. What "en masse" actually means is
beyond me, since I am yet to work out the
true rules regarding instantation. But what I can say is that having a template
such as the following is totally
f**cked:

    template TransformedRange(R, F, T) { class TransformedRange
     : public NotionalRange!(T)
    {
      . . .
        this(R r, F f)
        {
            m_r = r.dup;
            m_f = f;
        }
        this(R r)
        {
            m_r = r.dup;
            m_f = new filter_type();
        }

Without worrying too much about the details of what a TransformedRange does,
the problem is pretty obvious. If F does
not have a default constructor, one cannot instantiate TransformedRange even in
the case where the single parameter ctor
*is never called*!

I'm yet to work out a workaround to this one - all I do at the moment is not
use certain operations with certain
container instantiations in the test programs. Woo hoo!

3. There's no implicit instantiation. This has *massive* consequences,
including:

- We can't have iterators (as noted above). This is fine for reading from
containers, but consider how we might
"generically" insert into container ranges in the same (or any analogous) way
as is the case with STL.

- In order to support the parameterisable interface (e.g. IContainer!(int))
described above, there needs to be a common
way to manipulate built-in types, objects and structs. For some things, one can
use traits, for others, overloaded
functions. Alas, there seems to be no way to discriminate structs via
overloaded functions. Hence, currently the DTL
containers do not support polymorphic interfaces when storing structs.
Naturally, this has much wider consequences

4. D's import stuff just blows my mind! As a related issue to the boxing
utility class described above, I'm running into
conflicts between the toString() functions in std.string and my box and boxutil
modules. I've tried all kinds of use of
private imports, to no avail. I concede that this might be my fault, and I
might have just failed to grok D's import
rules, but as it currently looks to me, it looks bad and stupid.


So, what do we do about this? Walter's stock response is to boil it down, but
it's either impossible to do so with
non-trivial projects such as DTL, or I'm simply not smart enough to do it.
Sure, one might argue that this is indicative
of a too-complex design, but I think that's crap: the concept is simple, and
the code is simple; it just doesn't work.
(Very similar techniques that I've experimented on in C++ work fine.)

Rather than having things boiled down, I think the compiler should be amended
to provide *copious" debugging
information, so we can email a dump of that to Walter, and which will be useful
to him. I don't know what that
information should be, but I know that it's simply not practical for me, or
anyone else, to "boil down" these
precipitating bugs when they only manifest in highly complex code. I do know
that it's totally bogus for us to be made
to feel guilty for not having the time or the talent to do this boiling down. I
for one am investing a large amount of
time to the D cause, with all the consequences wrt to more materially lucrative
activities. Since I find that boiling
down the code generally takes me at least as long to write the code in the
first place, I just don't see that it's
justified that I should do the boiling as well. Give me a -dump switch, and I
can send it all off to Walter.

Anyway, that's just the compiler, and my main problem is with the language. I'm
coming to the conclusion that D either
will never be suitable for generic programming, or such suitability is years
away. Given that, my aims for DTL are
starting to seem naive at best, unattainable at worst.

What's to be done? Well, one might say let's just have vanilla containers, and
skip all the transformation stuff. That's
fine, but then where're the generics? We can't have algorithms, remember,
because we've not got implicit instantiation!
The only remaining way to be generic is to follow the Java-route, and go with
polymorphic container interfaces, but (i)
they can't contain structures, and (ii) we're in Java-la-la land where
everything has to be downcast. Yeuch! Even if we
can get the compiler to accept parameterisable container interfaces, it's still
a runtme indirection, with the
concomitant efficiency costs.

So please, someone enlighten me (since I am quite prepared to believe I've
missed something simple and obvious here):
how can we do generic programming in D?

Matthew

If all this sounds like I'm down on D, and I've spent several weeks - actually
it's several months, but who's
counting? - working against all these issues and more, then you'd be about spot
on. I can't remember being this
technically frustrated in the last 10 years, and I'm generally known as a
practical pragmatist! :-(
Jul 29 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceaeid$18ir$1 digitaldaemon.com>, Matthew says...
I'm almost at the end of my tether here. There are so many problems with the
language/compiler, that I'm wondering
seriously whether it's going to be possible to have a decent generic template
library.

This is a real bummer. I was rather hoping that D would end up being /better/ than C++. If that's not to be so, it's a little disheartening. Arcane Jill
Jul 29 2004
parent Sean Kelly <sean f4.ca> writes:
In article <ceamcj$1bco$1 digitaldaemon.com>, Arcane Jill says...
In article <ceaeid$18ir$1 digitaldaemon.com>, Matthew says...
I'm almost at the end of my tether here. There are so many problems with the
language/compiler, that I'm wondering
seriously whether it's going to be possible to have a decent generic template
library.

This is a real bummer. I was rather hoping that D would end up being /better/ than C++. If that's not to be so, it's a little disheartening.

I think it still will, but the language may have to become a bit more complicated to do so (which is a bit contrary to its original design parameters). It's probably a good time to start taking a critical look at the language and see if we can find holes in its capabilities. It would be nice if the issue were just one of mindset and not of feature support. Sean
Jul 29 2004
prev sibling next sibling parent h3r3tic <h3r3tic_member pathlink.com> writes:
Looks like we wont see D 1.0 anytime soon
*runs away crying*
Jul 29 2004
prev sibling next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
 Alternatively, we can also support the old-style Java approach, whereby
 one can specify a base interface, e.g.
 
     void dump(IObjectContainer c)
     {
         for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
         {
             printf("%.*s ", e.nextElement().toString());
         }
         printf("\n");
     }
 
     alias    Vector!(int, IObjectContainer)   
     int_vector_IObjectContainer_t;
     alias    List!(int, IObjectContainer)       
     int_list_IObjectContainer_t;
 
     int_vector_IObjectContainer_t   v = new ...;
     int_list_IObjectContainer_t        l = new ...;
 
     dump(v);
     dump(l);

As you say the Enumeration interface is old-style. It's the equivalent to porting Winows 3.1 to unix or something. Why not support "new-style" Java Iterators? I'm guessing that since the new-style is named Iterator it is too close to STL iterators. If that is the reason I'd look for another name - not that I have any off the top of my head. Or maybe just take the methods of Iterator and call it IList or something. In general, though, I don't think we should be saddled with Java's historical baggage.
Jul 29 2004
parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Ben Hinkle" <bhinkle4 juno.com> wrote in message
news:ceaqp7$1d2b$1 digitaldaemon.com...
 Alternatively, we can also support the old-style Java approach, whereby
 one can specify a base interface, e.g.

     void dump(IObjectContainer c)
     {
         for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
         {
             printf("%.*s ", e.nextElement().toString());
         }
         printf("\n");
     }

     alias    Vector!(int, IObjectContainer)
     int_vector_IObjectContainer_t;
     alias    List!(int, IObjectContainer)
     int_list_IObjectContainer_t;

     int_vector_IObjectContainer_t   v = new ...;
     int_list_IObjectContainer_t        l = new ...;

     dump(v);
     dump(l);

As you say the Enumeration interface is old-style. It's the equivalent to porting Winows 3.1 to unix or something. Why not support "new-style" Java Iterators? I'm guessing that since the new-style is named Iterator it is too close to STL iterators. If that is the reason I'd look for another name - not that I have any off the top of my head. Or maybe just take the methods of Iterator and call it IList or something. In general, though, I don't think we should be saddled with Java's historical baggage.

First: this stuff was worked out and implemented months ago, and in a single day. It costs nothing when you don't use it, and there are limited circumstances (e.g. Message boards) where such functionality is exactly right, so what's the beef? Second: you don't seem to have read the full post. The very next thing I discussed was the new Java-style parameterised interfaces, which is also already implemented, and seemingly supported by the language; just not by the compiler. Please have another look, and let me know if that was not clear.
Jul 29 2004
parent Ben Hinkle <bhinkle4 juno.com> writes:
Matthew wrote:

 
 "Ben Hinkle" <bhinkle4 juno.com> wrote in message
 news:ceaqp7$1d2b$1 digitaldaemon.com...
 Alternatively, we can also support the old-style Java approach, whereby
 one can specify a base interface, e.g.

     void dump(IObjectContainer c)
     {
         for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
         {
             printf("%.*s ", e.nextElement().toString());
         }
         printf("\n");
     }

     alias    Vector!(int, IObjectContainer)
     int_vector_IObjectContainer_t;
     alias    List!(int, IObjectContainer)
     int_list_IObjectContainer_t;

     int_vector_IObjectContainer_t   v = new ...;
     int_list_IObjectContainer_t        l = new ...;

     dump(v);
     dump(l);

As you say the Enumeration interface is old-style. It's the equivalent to porting Winows 3.1 to unix or something. Why not support "new-style" Java Iterators? I'm guessing that since the new-style is named Iterator it is too close to STL iterators. If that is the reason I'd look for another name - not that I have any off the top of my head. Or maybe just take the methods of Iterator and call it IList or something. In general, though, I don't think we should be saddled with Java's historical baggage.

First: this stuff was worked out and implemented months ago, and in a single day. It costs nothing when you don't use it, and there are limited circumstances (e.g. Message boards) where such functionality is exactly right, so what's the beef?

My beef is just with the choice of iterface to support. Sun's Javadoc for Enumeration says "NOTE: The functionality of this interface is duplicated by the Iterator interface. In addition, Iterator adds an optional remove operation, and has shorter method names. New implementations should consider using Iterator in preference to Enumeration."
 Second: you don't seem to have read the full post. The very next thing I
 discussed was the new Java-style parameterised interfaces, which is also
 already implemented, and seemingly supported by the language; just not by
 the compiler. Please have another look, and let me know if that was not
 clear.

umm - ok. I'll read it more carefully. I only saw Enumeration examples and no Iterator examples so I didn't realize you had also gotten Iterator ported. Maybe it's best for me to wait and see what is in there instead of guessing.
Jul 29 2004
prev sibling next sibling parent reply J C Calvarese <jcc7 cox.net> writes:
In article <ceaeid$18ir$1 digitaldaemon.com>, Matthew says...
So, what do we do about this? Walter's stock response is to boil it down, but
it's either impossible to do so with
non-trivial projects such as DTL, or I'm simply not smart enough to do it.
Sure, one might argue that this is indicative
of a too-complex design, but I think that's crap: the concept is simple, and
the code is simple; it just doesn't work.
(Very similar techniques that I've experimented on in C++ work fine.)

If you're unable (or unwilling) to boil it down for Walter, maybe someone else is. (I've worked on shortening problem code from Mango a while back.) If you post some code that produces a compiler error, I'll look at making it Walter-sized. jcc7
Jul 29 2004
parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"J C Calvarese" <jcc7 cox.net> wrote in message
news:ceb3ui$1gfa$1 digitaldaemon.com...
 In article <ceaeid$18ir$1 digitaldaemon.com>, Matthew says...
So, what do we do about this? Walter's stock response is to boil it down, but
it's either impossible to do so with
non-trivial projects such as DTL, or I'm simply not smart enough to do it.
Sure, one might argue that this is


of a too-complex design, but I think that's crap: the concept is simple, and
the code is simple; it just doesn't


(Very similar techniques that I've experimented on in C++ work fine.)

If you're unable (or unwilling) to boil it down for Walter, maybe someone else is. (I've worked on shortening problem code from Mango a while back.) If you post some code that produces a compiler error, I'll look at making it Walter-sized.

Good to know. I guess a lot of people will be able to help out in a variety of ways as soon as I damn-well post something. ;)
Jul 30 2004
prev sibling next sibling parent reply teqDruid <me teqdruid.com> writes:
I've read over your rant.  I'll not pretend to understand all of it, and I
don't have any suggestions for your specific problems, except to that the
Java route is not the way to go. I'm a Java guy, and I hate the Java
collections API.

Whenever I work on an open-source project, I put my code in a publicly
accessable repository (a subversion repository, specifically) and I
generally commit my code to it after each new change (even small changes,
like a since bug fix) and generally before I stop for the day.  This way,
I can get input from others, not that there are many (hell, let's face it:
any) followers of my projects.  (Plus, the code gets backed up, but that's
OT.) It's kind of a "release early, release often" approach (btw, if you
haven't read ESR's "The Cathedral and the Bazaar", it's a good read.) I
think your decision to release what you've got in a day or two is a good
one. It's always good to have more people reading the code. The more
perspectives one can get input from, the more successful a project will
be, in my opinion.  I would even encourage you to put the project on
dsource, and commit to the repository often.  I know I'd definately be
more help giving advice (or even patches) if I could see the entire source.

On a very positive note, from what I understood of your post, it appears
to me that should the language/compiler problems get worked out, DTL will
be a very powerful, robust tool.  I am most impressed.

As always, I salute you for your effort, and intellect.
John

I hope to see Walter reply on this one.  I will be throughly disappointed
should he not.

On Thu, 29 Jul 2004 19:04:11 +1000, Matthew wrote:

 I'm almost at the end of my tether here. There are so many problems with the
language/compiler, that I'm wondering
 seriously whether it's going to be possible to have a decent generic template
library.
 ...
 If all this sounds like I'm down on D, and I've spent several weeks -
 actually it's several months, but who's counting? - working against all
 these issues and more, then you'd be about spot on. I can't remember
 being this technically frustrated in the last 10 years, and I'm
 generally known as a practical pragmatist! :-(

Jul 29 2004
parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"teqDruid" <me teqdruid.com> wrote in message
news:pan.2004.07.29.15.13.34.724295 teqdruid.com...
 I've read over your rant.  I'll not pretend to understand all of it, and I
 don't have any suggestions for your specific problems, except to that the
 Java route is not the way to go. I'm a Java guy, and I hate the Java
 collections API.

 Whenever I work on an open-source project, I put my code in a publicly
 accessable repository (a subversion repository, specifically) and I
 generally commit my code to it after each new change (even small changes,
 like a since bug fix) and generally before I stop for the day.  This way,
 I can get input from others, not that there are many (hell, let's face it:
 any) followers of my projects.  (Plus, the code gets backed up, but that's
 OT.) It's kind of a "release early, release often" approach (btw, if you
 haven't read ESR's "The Cathedral and the Bazaar", it's a good read.) I
 think your decision to release what you've got in a day or two is a good
 one. It's always good to have more people reading the code. The more
 perspectives one can get input from, the more successful a project will
 be, in my opinion.  I would even encourage you to put the project on
 dsource, and commit to the repository often.  I know I'd definately be
 more help giving advice (or even patches) if I could see the entire source.

The problem I've had is that the things that have held me up are not things that anyone other than Walter could help me with, since they've been compiler bugs and (missing) language features. Were I to have posted it, people would've seen nothing more than non-compilable, or crashing, code The actual containment aspects of the library were implemented in March, and I've been fighting the compiler/language on the "advanced" issues ever since. I certainly see now that I should have released those parts, as I plan to this w/e, some time ago, but all along I've expected that the compiler/language enhancements were days away, rather than the months it's taken. This is not a gripe at Walter, more my having been too optimistic.
 On a very positive note, from what I understood of your post, it appears
 to me that should the language/compiler problems get worked out, DTL will
 be a very powerful, robust tool.  I am most impressed.

I am very confident that will be the case.
 As always, I salute you for your effort, and intellect.

Well, I'll accept the former. Not sure I deserve the latter. <G>
 John

 I hope to see Walter reply on this one.  I will be throughly disappointed
 should he not.

 On Thu, 29 Jul 2004 19:04:11 +1000, Matthew wrote:

 I'm almost at the end of my tether here. There are so many problems with the
language/compiler, that I'm wondering
 seriously whether it's going to be possible to have a decent generic template
library.
 ...
 If all this sounds like I'm down on D, and I've spent several weeks -
 actually it's several months, but who's counting? - working against all
 these issues and more, then you'd be about spot on. I can't remember
 being this technically frustrated in the last 10 years, and I'm
 generally known as a practical pragmatist! :-(


Jul 30 2004
parent teqDruid <me teqdruid.com> writes:
On Fri, 30 Jul 2004 17:53:20 +1000, Matthew wrote:

 
 "teqDruid" <me teqdruid.com> wrote in message
news:pan.2004.07.29.15.13.34.724295 teqdruid.com...
 I've read over your rant.  I'll not pretend to understand all of it, and I
 don't have any suggestions for your specific problems, except to that the
 Java route is not the way to go. I'm a Java guy, and I hate the Java
 collections API.

 Whenever I work on an open-source project, I put my code in a publicly
 accessable repository (a subversion repository, specifically) and I
 generally commit my code to it after each new change (even small changes,
 like a since bug fix) and generally before I stop for the day.  This way,
 I can get input from others, not that there are many (hell, let's face it:
 any) followers of my projects.  (Plus, the code gets backed up, but that's
 OT.) It's kind of a "release early, release often" approach (btw, if you
 haven't read ESR's "The Cathedral and the Bazaar", it's a good read.) I
 think your decision to release what you've got in a day or two is a good
 one. It's always good to have more people reading the code. The more
 perspectives one can get input from, the more successful a project will
 be, in my opinion.  I would even encourage you to put the project on
 dsource, and commit to the repository often.  I know I'd definately be
 more help giving advice (or even patches) if I could see the entire source.

The problem I've had is that the things that have held me up are not things that anyone other than Walter could help me with, since they've been compiler bugs and (missing) language features. Were I to have posted it, people would've seen nothing more than non-compilable, or crashing, code

Yes, but you said that you don't have time to "boil-down" code for submission. Merely posting broken code, I'm sure people would be willing to help prepare bug reports and testcases for Walter.
Jul 30 2004
prev sibling next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
 - In order to support the parameterisable interface (e.g.

 way to manipulate built-in types, objects and structs. For some things,

 functions. Alas, there seems to be no way to discriminate structs via

 containers do not support polymorphic interfaces when storing structs.

Could TypeInfo's help here? Structs don't currently have TypeInfo so it won't solve the struct issue but Walter has said he knows TypeInfos for structs needs improvement so eventually it should get fixed. I've been using TypeInfo for compare, swap, equals and getHash and found them very useful.
Jul 29 2004
next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <ceb6ur$1hn3$1 digitaldaemon.com>, Ben Hinkle says...
Could TypeInfo's help here? Structs don't currently have TypeInfo so it
won't solve the struct issue but Walter has said he knows TypeInfos for
structs needs improvement so eventually it should get fixed. I've been using
TypeInfo for compare, swap, equals and getHash and found them very useful.

I suggested the same thing, but TypeInfo fails in eone important manner: TypeInfo is evaluated at run-time. I'm only speculating, but I have a feeling that we may also need some compile-time means for differentiating types. C++ doesn't have this particular problem because structs and classes are the same thing. Sean
Jul 29 2004
parent pragma <EricAnderton at yahoo dot com> <pragma_member pathlink.com> writes:
In article <ceb98u$1iff$1 digitaldaemon.com>, Sean Kelly says...
In article <ceb6ur$1hn3$1 digitaldaemon.com>, Ben Hinkle says...
Could TypeInfo's help here? Structs don't currently have TypeInfo so it
won't solve the struct issue but Walter has said he knows TypeInfos for
structs needs improvement so eventually it should get fixed. I've been using
TypeInfo for compare, swap, equals and getHash and found them very useful.

I suggested the same thing, but TypeInfo fails in eone important manner: TypeInfo is evaluated at run-time. I'm only speculating, but I have a feeling that we may also need some compile-time means for differentiating types. C++ doesn't have this particular problem because structs and classes are the same thing.

I don't profess to have done anywhere near as much template programming as Matthew, but I've run into some quirks in D that force me to write some really painful runarounds. That aside, it _is_ possible to use templates based on TypeInfo to get the compile-time typing you're after. template Foo(TI: TypeInfo,T){ /* T is a primitive type */ } template Foo(TI: TypeInfoClass,T){ /* T is a class */ } template Bar(T: void){ /* T is void */ } template Bar(T: Object){ /* T is a class */ } template Bar(T){ /* T is a primitive */ } // examples Foo!(typeid(int),int); Foo!(typeid(MyClass),MyClass); Bar(void); Bar(MyClass); Bar(int); - Pragma
Jul 29 2004
prev sibling parent "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Ben Hinkle" <bhinkle mathworks.com> wrote in message
news:ceb6ur$1hn3$1 digitaldaemon.com...
 - In order to support the parameterisable interface (e.g.

 way to manipulate built-in types, objects and structs. For some things,

 functions. Alas, there seems to be no way to discriminate structs via

 containers do not support polymorphic interfaces when storing structs.

Could TypeInfo's help here? Structs don't currently have TypeInfo so it won't solve the struct issue but Walter has said he knows TypeInfos for structs needs improvement so eventually it should get fixed. I've been using TypeInfo for compare, swap, equals and getHash and found them very useful.

Very probably. I confess my experience with/understanding of structs is not what you'd call comprehensive. :)
Jul 29 2004
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <ceaeid$18ir$1 digitaldaemon.com>, Matthew says...
4. Iterator based approach. There are two reasons why STL-iterators are not
possible in D: (i) there is no implicit
instantiation, and (ii) there is no operator *().

In my more evil moments I've considered requesting an overloaded dot operator for much the same reason operator-> can be overloaded in C++. I find it unfortunate that smart pointers can't really be implemented in D either, even though GC makes them far less necessary.
2. Templates in D are instantiated en masse. What "en masse" actually means is
beyond me, since I am yet to work out the
true rules regarding instantation. But what I can say is that having a template
such as the following is totally
f**cked:

    template TransformedRange(R, F, T) { class TransformedRange
     : public NotionalRange!(T)
    {
      . . .
        this(R r, F f)
        {
            m_r = r.dup;
            m_f = f;
        }
        this(R r)
        {
            m_r = r.dup;
            m_f = new filter_type();
        }

Without worrying too much about the details of what a TransformedRange does,
the problem is pretty obvious. If F does
not have a default constructor, one cannot instantiate TransformedRange even in
the case where the single parameter ctor
*is never called*!

Your example is confusing. I don't see "new F()" anywhere. Why the need for a default constructor? But this raises an interesting point. One useful but little mentioned feature of C++ is that primitive types support default constructor semantics just like user defined classes do. For generic programming it might be nice to have the same thing: MyClass c = new MyClass(); int i = new int(); // equivalent to "int i;" This makes the difference between pointers and nonpointers a tad confusing but it would make life much easier in some cases.
3. There's no implicit instantiation. This has *massive* consequences,
including:

- In order to support the parameterisable interface (e.g. IContainer!(int))
described above, there needs to be a common
way to manipulate built-in types, objects and structs. For some things, one can
use traits, for others, overloaded
functions. Alas, there seems to be no way to discriminate structs via
overloaded functions. Hence, currently the DTL
containers do not support polymorphic interfaces when storing structs.

See my comment above. This is pretty important. We should be able to use the same semantics for all types in D and be able to distinguish between them. TypeInfo is a good start, though someone mentioned that structs and pointers have the same base TypeInfo class?
4. D's import stuff just blows my mind! As a related issue to the boxing
utility class described above, I'm running into
conflicts between the toString() functions in std.string and my box and boxutil
modules.

I ran into this problem when trying to import std.ctype and std.c.string into the same module. I ended up just adding extern declarations for the C functions I wanted, but it's an outstanding issue. IMO an import should import everything it can and leave conflicting symbols out. Then they could still be referenced via std.string.strcmp or whatever. Perhaps this is a good instance where a compiler warning would be useful? I've tried all kinds of use of
private imports, to no avail. I concede that this might be my fault, and I
might have just failed to grok D's import
rules, but as it currently looks to me, it looks bad and stupid.

I haven't found any way around this either, but I think this may be more a bug with the import functionality than any language deficiency. It's something I haven't taken the time to experiment with sufficiently.
Rather than having things boiled down, I think the compiler should be amended
to provide *copious" debugging
information, so we can email a dump of that to Walter, and which will be useful
to him. I don't know what that
information should be, but I know that it's simply not practical for me, or
anyone else, to "boil down" these
precipitating bugs when they only manifest in highly complex code.

That's a handy idea, provided doing so doesn't take up weeks of Walter's time to add.
Anyway, that's just the compiler, and my main problem is with the language. I'm
coming to the conclusion that D either
will never be suitable for generic programming, or such suitability is years
away. Given that, my aims for DTL are
starting to seem naive at best, unattainable at worst.

I think some language issues may just need to be refined. Implicit instantiation is a big one for me, though I know that it is not a simple feature to add. Ideally, I would like to do everything GP-wise in D that I can in C++. Perhaps a set of examples of what works and what doesn't followed by a discussion of language features D may be lacking? In some respects I've found D more capable (alias template parameters) and in others, less.
 We can't have algorithms, remember, because we've not got implicit
instantiation!

Well, we can in some cases because we've got typeof, it just means more typing. But I'm sure there are cases where types are determined in odd ways in C++ that a wrapper template in D couldn't work around.
So please, someone enlighten me (since I am quite prepared to believe I've
missed something simple and obvious here):
how can we do generic programming in D?

Let's try and work it out. I'll admit I've been focusing on other things until now. Another day and I'll be finished with unFormat and I'll be able to play with a new project. Perhaps trying to implement <algorithm> and <functional> would be a good place to start. At the very least it would help to determine just what D can and can't do so far as templates are concerned. The iterator thing is a tough one though, because so much depends on that. Is there truly no way to generate them? I had played with creating a list type and thought I'd got the basics of one working, though I didn't experiment much with primitves vs. structs vs. classes. Sean
Jul 29 2004
next sibling parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Sean Kelly" <sean f4.ca> wrote in message
news:ceb8vu$1ibm$1 digitaldaemon.com...
 In article <ceaeid$18ir$1 digitaldaemon.com>, Matthew says...
4. Iterator based approach. There are two reasons why STL-iterators are not
possible in D: (i) there is no implicit
instantiation, and (ii) there is no operator *().

In my more evil moments I've considered requesting an overloaded dot operator for much the same reason operator-> can be overloaded in C++. I find it unfortunate that smart pointers can't really be implemented in D either, even though GC makes them far less necessary.

He he. Actually, I like the fact that smart pointers are not really possible in D. I pretty much hate smart pointers in C++. (There was going to be a chapter in "Imperfect C++" on smart pointers, but I was told by the publisher to "Cut! Cut! Cut!", so it's one of the topics that got shelved. I do discuss some of the more heinous abuses, such as overloading operator &. Anyway, Maybe I'll discuss the issue in my next C++ book, although that's not going to be done until next year.) Anyway, to the point: since pointers are second-class citizens in D, I really can't see the point of smart-pointers at all, and I think it's actually a net boon that we won't have them.
2. Templates in D are instantiated en masse. What "en masse" actually means is
beyond me, since I am yet to work out


true rules regarding instantation. But what I can say is that having a template
such as the following is totally
f**cked:

    template TransformedRange(R, F, T) { class TransformedRange
     : public NotionalRange!(T)
    {
      . . .
        this(R r, F f)
        {
            m_r = r.dup;
            m_f = f;
        }
        this(R r)
        {
            m_r = r.dup;
            m_f = new filter_type();
        }

Without worrying too much about the details of what a TransformedRange does,
the problem is pretty obvious. If F does
not have a default constructor, one cannot instantiate TransformedRange even in
the case where the single parameter


*is never called*!

Your example is confusing. I don't see "new F()" anywhere. Why the need for a default constructor?

My bad. filter_type is an alias of F, which I elided for clarity (!), but left one of them in. It should be
        this(R r)
        {
            m_r = r.dup;
            m_f = new F();
        }


 But this raises an interesting point.  One useful but little mentioned feature
 of C++ is that primitive types support default constructor semantics just like
 user defined classes do.  For generic programming it might be nice to have the
 same thing:

It certainly would. I assumed they did. Is that not the case in D?
 MyClass c = new MyClass();
 int i = new int(); // equivalent to "int i;"

 This makes the difference between pointers and nonpointers a tad confusing but
 it would make life much easier in some cases.

Ah, no. It will create one on the heap. Nasty!
3. There's no implicit instantiation. This has *massive* consequences,
including:

- In order to support the parameterisable interface (e.g. IContainer!(int))
described above, there needs to be a


way to manipulate built-in types, objects and structs. For some things, one can
use traits, for others, overloaded
functions. Alas, there seems to be no way to discriminate structs via
overloaded functions. Hence, currently the DTL
containers do not support polymorphic interfaces when storing structs.

See my comment above. This is pretty important. We should be able to use the same semantics for all types in D and be able to distinguish between them.

Agreed. I'm sure we can sort this out in the long run. I suspect that as soon as Walter looks at the issue he'll point out some bit of template arcana that I should have been applying.
 TypeInfo is a good start, though someone mentioned that structs and pointers
 have the same base TypeInfo class?

I know virtually nothing about TypeInfo. I guess I should correct that. :)
4. D's import stuff just blows my mind! As a related issue to the boxing
utility class described above, I'm running


conflicts between the toString() functions in std.string and my box and boxutil
modules.

I ran into this problem when trying to import std.ctype and std.c.string into the same module. I ended up just adding extern declarations for the C functions I wanted, but it's an outstanding issue. IMO an import should import everything it can and leave conflicting symbols out. Then they could still be referenced via std.string.strcmp or whatever. Perhaps this is a good instance where a compiler warning would be useful? I've tried all kinds of use of
private imports, to no avail. I concede that this might be my fault, and I
might have just failed to grok D's import
rules, but as it currently looks to me, it looks bad and stupid.

I haven't found any way around this either, but I think this may be more a bug with the import functionality than any language deficiency. It's something I haven't taken the time to experiment with sufficiently.
Rather than having things boiled down, I think the compiler should be amended
to provide *copious" debugging
information, so we can email a dump of that to Walter, and which will be useful
to him. I don't know what that
information should be, but I know that it's simply not practical for me, or
anyone else, to "boil down" these
precipitating bugs when they only manifest in highly complex code.

That's a handy idea, provided doing so doesn't take up weeks of Walter's time to add.
Anyway, that's just the compiler, and my main problem is with the language. I'm
coming to the conclusion that D


will never be suitable for generic programming, or such suitability is years
away. Given that, my aims for DTL are
starting to seem naive at best, unattainable at worst.

I think some language issues may just need to be refined. Implicit instantiation is a big one for me, though I know that it is not a simple feature to add. Ideally, I would like to do everything GP-wise in D that I can in C++. Perhaps a set of examples of what works and what doesn't followed by a discussion of language features D may be lacking? In some respects I've found D more capable (alias template parameters) and in others, less.

Same here.
 We can't have algorithms, remember, because we've not got implicit
instantiation!

Well, we can in some cases because we've got typeof, it just means more typing. But I'm sure there are cases where types are determined in odd ways in C++ that a wrapper template in D couldn't work around.

I think so.
So please, someone enlighten me (since I am quite prepared to believe I've
missed something simple and obvious here):
how can we do generic programming in D?

Let's try and work it out. I'll admit I've been focusing on other things until now. Another day and I'll be finished with unFormat and I'll be able to play with a new project. Perhaps trying to implement <algorithm> and <functional> would be a good place to start. At the very least it would help to determine just what D can and can't do so far as templates are concerned. The iterator thing is a tough one though, because so much depends on that. Is there truly no way to generate them? I had played with creating a list type and thought I'd got the basics of one working, though I didn't experiment much with primitves vs. structs vs. classes.

We need some kind of implicit instantiation, however limited, in order to get true generic programming with iterators. Without it, we're going to have to have iterator interfaces, and be accessing elements polymorphically, which is not generic programming IME.
Jul 30 2004
parent Sean Kelly <sean f4.ca> writes:
In article <cecvci$2as7$3 digitaldaemon.com>, Matthew says...
"Sean Kelly" <sean f4.ca> wrote in message
news:ceb8vu$1ibm$1 digitaldaemon.com...
 MyClass c = new MyClass();
 int i = new int(); // equivalent to "int i;"

 This makes the difference between pointers and nonpointers a tad confusing but
 it would make life much easier in some cases.

Ah, no. It will create one on the heap. Nasty!

Yeah the syntax kind of stinks. I suppose we could fix this now with templates (see below) but it would still be nice to have some sort of language support for this, though I don't have any ideas offhand. Perhaps this? T val = T.new;
  We should be able to use the
 same semantics for all types in D and be able to distinguish between them.

Agreed. I'm sure we can sort this out in the long run. I suspect that as soon as Walter looks at the issue he'll point out some bit of template arcana that I should have been applying.

This was meant for generating copies but it could be adapted for plain ol' construction. # template gen( Ty ) # { # Ty copy( Ty val ) # { # return val; # } # } # # # template gen( Ty : Object ) # { # Ty copy( Ty val ) # { # return new Ty( val ); # } # } The new class and struct keywords Walter suggestede would make this even more foolproof. Sean
Jul 30 2004
prev sibling parent Regan Heath <regan netwin.co.nz> writes:
On Thu, 29 Jul 2004 16:35:10 +0000 (UTC), Sean Kelly <sean f4.ca> wrote:
<snip>
 But this raises an interesting point.  One useful but little mentioned 
 feature
 of C++ is that primitive types support default constructor semantics 
 just like
 user defined classes do.  For generic programming it might be nice to 
 have the
 same thing:

 MyClass c = new MyClass();
 int i = new int(); // equivalent to "int i;"

 This makes the difference between pointers and nonpointers a tad 
 confusing but
 it would make life much easier in some cases.

Or, go the other way, as Arcane Jill has suggested in the past. MyClass c(1,2,3); //equivalent to "MyClass c = new MyClass(1,2,3); MyStruct s(4,5,6); //give us struct constructors :) int i(1); //equivalent to "int i = 1;" The only confusion I see here is that the class is on the heap and the others on the stack so they're different but they don't look it. Then again who said you have to get rid of what we currently have, all we really need to do is add this option. Regan -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 31 2004
prev sibling next sibling parent reply Sha Chancellor <schancel pacific.net> writes:
Matthew,
 
   Compiler bugs aside, you forté make a few silly arguments.  Such as, 
that you can't support iterators because you can't overload the * 
operator.  What kind of BS is that?   

Overloading operators for no apparent good reason is STLs forté.  Obj-C 
and Cocoa have lots of templates and it doesn't have any operator 
overloading what-so-ever.

STLs and C++ use operator overloading incessantly  because somehow their 
designers thinks it looks cool.   IT confuses whether or not an iterator 
is a pointer or an object.  Why not just have a real function call:  
Iterator.getMyFuckingInstanceNow()

No instead we must confuse things by going (*Iterator)

Iterators in STL suck, plain and simple.  Most of C++'s standard 
template library is overly complicated and obtuse for the job.  Not to 
mention buggy between implementations.


As for implicit instantation,  Why is this required to make an iterator?

Maybe I'm missing the boat, but what's wrong with something like this:

class List(T) {
   alias ListIterator(T) iterator;

   ...
}

List!(int).iterator iter;
// Hell you could probably even take your existing instance and do:
// Foo.iterator iter;  In fact you should be able to but it seems you 
can't  This would allow you to also do aliases for stuff like:

iter.BaseType t; and do your generic manipulation functions.
or List.BaseType; etc.  And have aliases in your class.   

I say we bitch at walter.

for( iter = myList.begin();  iter != myList.end(); iter++ )
{
   printf(iter.getMyFuckingInstanceNow().toString());
}

This is nearly identical to what C++ does, and i really don't see why it 
wouldn't work.  You could probably leave the instance-getter function 
name a little shorter though.

Maybe you could expand as to why this sucks? (Besides that it's similar 
to STL which sucks)


And as for your template example:


#    template dump(T) {
#    void dump(IContainer!(T) c)
#    {
#        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
#        {
#
#        }
#    }
#    }
#
#    List!(int, IContainer!(int))           l = new ...;
#
#    dump(l);

For one, making those aliases all over is just as ugly and hard to 
remember as just specifying it.  Second:

You're passing in l of type List!(int, IContainer!(int)), and then 
proceed to do this with it:

IContainer!(List!(int, IContainer!(int))) c.

Is that really what you were intending to do?

It seems like templates should support something along these lines:

#   template dump( T : IContainer!(T) ) {
#      void dump( IContainer!(t) ) {
#      }
#   }

Although I seriously doubt that works as it is.
Jul 29 2004
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Sha Chancellor" <schancel pacific.net> wrote in message
news:schancel-C6DC99.10061329072004 digitalmars.com...
 Matthew,

    Compiler bugs aside, you forté make a few silly arguments.  Such as,
 that you can't support iterators because you can't overload the *
 operator.  What kind of BS is that?

I said "Neither of these preclude the creation of iterators - and I've done a little of that - ". Maybe you should have read it more thoroughly. :-(
 Overloading operators for no apparent good reason is STLs forté.  Obj-C
 and Cocoa have lots of templates and it doesn't have any operator
 overloading what-so-ever.

 STLs and C++ use operator overloading incessantly  because somehow their
 designers thinks it looks cool.

Wrong. (Or half wrong anyway). They overload the operators for conformance with pointers. Surely you know/recognise that a pointer is a RandomAccess iterator? It's been my opinion - espoused often on the D ngs over the last year or so - that we don't need to directly support pointers as "first class" members of the enumerable club in D, since D's arrays (and any UDTs) are already enumerable via foreach(). This has been one of the motivating factors in my design toward a foreach-oriented library, with the consequences that two of the four of the enumeration "modes" / "models" in DTL are based on foreach.
   IT confuses whether or not an iterator
 is a pointer or an object.  Why not just have a real function call:

Pointers, as explained above.
 Iterator.getMyFuckingInstanceNow()

Yes. The iterators I've played with have taken this approach.
 No instead we must confuse things by going (*Iterator)

 Iterators in STL suck, plain and simple.  Most of C++'s standard
 template library is overly complicated and obtuse for the job.  Not to
 mention buggy between implementations.

At the risk of pissing off YAA, I think this is naive. (Aside: pretty much everyone in the world apart from fellow Englishmen tend to get shirty if I mention the "n" word. For my part, it's much less offensive to have one's opinion called naive, than ill-informed, or half-considered, or plain dumb. So naive it is.) Yes, C++'s library is complex, and it's going to get a lot more so in C++-0.x. Yes, templates are hard to understand, and template implementations are virtually impossible not to become dialecticised (?). Consider, even though I value simplicity in templates more highly than pretty much any other aspect, I've done some reasonably complex things with STLSoft, and some people have commented that that stuff is pretty advanced/over-complicated/obtuse/cool/stupid. Point being, I understand the complex techniques I've used in STLSoft, but I cannot understand the implementation of even small Boost modules, for the same reason that some Boosters cannot understand STLSoft: C++ is too complex. But, C++ is powerful, and that's why we stick with it. To stand on the sidelines and just shout about how hard it is is plain silly, until something replaces it. I don't think I speak out of turn to say that Walter would like that to be D. To achieve that D *must* support the currently unmatched power of C++. The other issues are (i) whether it becomes as hard, and (ii) whether it engenders the confusing and disenfranchising dialectisation (?) of C++. I my opinion we can live with (i) if we must, because most developers are reasonably intelligent. But (ii) will kill it, because learning limitless dialects is incredibly irritating and a waste of effort. If D ends up being as big a bag of wild-woman's knitting as C++, then what's the motivation to move to D? [Apologies to any wild-women who have neat knitting.]
 As for implicit instantation,  Why is this required to make an iterator?

Sigh. Either I have all the didactic talent of your average post-graduate tutorial host, or you've not read my post correctly. What I said was that to use iterators generically, and without iterators sharing a common interface (e.g. IObjectEnumerator, or IEnumerator!(int)), we need implicit instantiation.
 Maybe I'm missing the boat, but what's wrong with something like this:

 class List(T) {
    alias ListIterator(T) iterator;

    ...
 }

 List!(int).iterator iter;
 // Hell you could probably even take your existing instance and do:
 // Foo.iterator iter;  In fact you should be able to but it seems you
 can't  This would allow you to also do aliases for stuff like:

 iter.BaseType t; and do your generic manipulation functions.
 or List.BaseType; etc.  And have aliases in your class.

This kind of thing is already done. One can (or at least could - I've changed a lot of things since April, when I last built and tested with -version=UseIterators) do things such as: List!(int) l = new ... for(List!(int).iterator b = l.begin(), e = l.end(); b != e; b.next()) { int i = b.value(); } But one cannot do something such as int j = accumulate(l.begin(), l.end(), 0); To do this, one has to provide the type(s), as in: int j = accumulate!(List!(int).iterator, int)(l.begin ...) [Actually, since all iterators must be classes, I've prescribed that they all have a standard set of member types, so it's possible to deduce the value type, as in: int j = accumulate!(List!(int).iterator)(l.begin ...) ] Given that the (currently) only useful mode for iterators is where the type is explicit, it's my opinion (and that of others, I think) that there's no benefit over foreach, hence the focus on foreach for the container enumeration, and the enumeration of transformed ranges, as in: foreach(int i; l) { . . . } and foreach(int i; l.select(IsOdd)) { . . . }
 I say we bitch at walter.

 for( iter = myList.begin();  iter != myList.end(); iter++ )
 {
    printf(iter.getMyFuckingInstanceNow().toString());
 }

 This is nearly identical to what C++ does, and i really don't see why it
 wouldn't work.

It can work. It does work. I've mentioned this many times on the NGs.
  You could probably leave the instance-getter function
 name a little shorter though.

D'ya think?
 Maybe you could expand as to why this sucks? (Besides that it's similar
 to STL which sucks)

Again, a pointless comment that suggests ignorance, or a fondness for glib statements at best. STL is a mostly marvellous idea/technology, but the bits of it that are bad are glaringly bad. It does not suck, it's the current best effort at powerful generic programming. Just because something has bad parts doesn't mean it sucks. If that's the case, then everything sucks. It's my hope and, I would think, that of most people in D, that DTL can be (nearly) as powerful as STL, but jettison most of the glaringly bad parts. I don't expect DTL to be without flaws, but it needs to be simpler, and less prone to spawning dialects, than STL. And it needs more power than the language currently provides.
 And as for your template example:


 #    template dump(T) {
 #    void dump(IContainer!(T) c)
 #    {
 #        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
 #        {
 #
 #        }
 #    }
 #    }
 #
 #    List!(int, IContainer!(int))           l = new ...;
 #
 #    dump(l);

 For one, making those aliases all over is just as ugly and hard to
 remember as just specifying it.  Second:

 You're passing in l of type List!(int, IContainer!(int)), and then
 proceed to do this with it:

 IContainer!(List!(int, IContainer!(int))) c.

 Is that really what you were intending to do?

I'm afraid I can't follow the last section. Can you rephrase?
 It seems like templates should support something along these lines:

 #   template dump( T : IContainer!(T) ) {
 #      void dump( IContainer!(t) ) {
 #      }
 #   }

 Although I seriously doubt that works as it is.

What's "t"? If you mean "T", then I presume the different you're providing is the constraint on T to be "in" an IContainer!(T)?? Doesn't the function signature already do that? Please elucidate.
Jul 29 2004
next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
news:cebp7v$1pt7$2 digitaldaemon.com...
 This kind of thing is already done. One can (or at least could - I've

 built and tested with -version=UseIterators) do things such as:

     List!(int)    l = new ...

     for(List!(int).iterator b = l.begin(), e = l.end(); b != e; b.next())
     {
         int i = b.value();
     }

 But one cannot do something such as

     int j = accumulate(l.begin(), l.end(), 0);

That's a compiler bug, now fixed. At least I think this should work now <g>.
Jul 29 2004
parent reply Sean Kelly <sean f4.ca> writes:
In article <cebrh0$1qnl$1 digitaldaemon.com>, Walter says...
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
news:cebp7v$1pt7$2 digitaldaemon.com...
 This kind of thing is already done. One can (or at least could - I've

 built and tested with -version=UseIterators) do things such as:

     List!(int)    l = new ...

     for(List!(int).iterator b = l.begin(), e = l.end(); b != e; b.next())
     {
         int i = b.value();
     }

 But one cannot do something such as

     int j = accumulate(l.begin(), l.end(), 0);

That's a compiler bug, now fixed. At least I think this should work now <g>.

Okay, now I'm confused. Matthew's example seems to be that implicit type discovery doesn't work with templates. Surely this can't be the bug that was fixed. Sean
Jul 29 2004
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Sean Kelly" <sean f4.ca> wrote in message
news:cebtmu$1ro6$1 digitaldaemon.com...
 In article <cebrh0$1qnl$1 digitaldaemon.com>, Walter says...
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
news:cebp7v$1pt7$2 digitaldaemon.com...
 This kind of thing is already done. One can (or at least could - I've

 built and tested with -version=UseIterators) do things such as:

     List!(int)    l = new ...

     for(List!(int).iterator b = l.begin(), e = l.end(); b != e; b.next())
     {
         int i = b.value();
     }

 But one cannot do something such as

     int j = accumulate(l.begin(), l.end(), 0);

That's a compiler bug, now fixed. At least I think this should work now <g>.

Okay, now I'm confused. Matthew's example seems to be that implicit type discovery doesn't work with templates. Surely this can't be the bug that was fixed.

I'm with Sean on this one. Since D doesn't have implicit instantion, how can an accumulate template be instantiated as shown above?
Jul 29 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
news:cec0fl$1t7c$1 digitaldaemon.com...
 I'm with Sean on this one. Since D doesn't have implicit instantion, how

 shown above?

Can you post a complete example, and I can verify whether it is fixed or not.
Jul 29 2004
parent reply Sean Kelly <sean f4.ca> writes:
Walter wrote:

 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
 news:cec0fl$1t7c$1 digitaldaemon.com...
 
I'm with Sean on this one. Since D doesn't have implicit instantion, how

shown above?

Can you post a complete example, and I can verify whether it is fixed or not.

Just to be pedantic: # template accumulate( Iter, ValT ) # { # ValT accumulate( Iter begin, Iter end, ValT val = ValT.init ) {} # } # # // what Matthew wants # char* b, e; # accumulate( b, e, 0 ); # # // what we've got to do # accumulate!(char*, char)( b, e, 0 ); So in this case it's not a bug but a language issue :) Sean
Jul 29 2004
next sibling parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Sean Kelly" <sean f4.ca> wrote in message
news:ceca7q$213u$1 digitaldaemon.com...
 Walter wrote:

 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
 news:cec0fl$1t7c$1 digitaldaemon.com...

I'm with Sean on this one. Since D doesn't have implicit instantion, how

shown above?

Can you post a complete example, and I can verify whether it is fixed or not.

Just to be pedantic: # template accumulate( Iter, ValT ) # { # ValT accumulate( Iter begin, Iter end, ValT val = ValT.init ) {} # } # # // what Matthew wants # char* b, e; # accumulate( b, e, 0 ); # # // what we've got to do # accumulate!(char*, char)( b, e, 0 ); So in this case it's not a bug but a language issue :)

Well, to be equally pedantic, I'm not too troubled about pointers, but in essence you're correct. I agree that it's not a bug. If I've given the impression that this is a bug, I'm sorry, because I see it entirely as a language deficiency, not a compiler bug
Jul 29 2004
parent reply Sean Kelly <sean f4.ca> writes:
Matthew wrote:
 
 Well, to be equally pedantic, I'm not too troubled about pointers, but in
essence you're correct.

Just wanted a simple example. The fact that the MSVC++ debugger chokes on debug symbols longer than 256 chars (and does this quite regularly) shows just how long template names can get ;) Sean
Jul 29 2004
parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
You talking about the 4786 issue, or something else?


"Sean Kelly" <sean f4.ca> wrote in message
news:ceci6c$243h$2 digitaldaemon.com...
 Matthew wrote:
 Well, to be equally pedantic, I'm not too troubled about pointers, but in
essence you're correct.

Just wanted a simple example. The fact that the MSVC++ debugger chokes on debug symbols longer than 256 chars (and does this quite regularly) shows just how long template names can get ;) Sean

Jul 30 2004
parent Sean Kelly <sean f4.ca> writes:
In article <ced03n$2b4a$1 digitaldaemon.com>, Matthew says...
You talking about the 4786 issue, or something else?

Yup. Only useful in that it shows that complete template names often get quite large, even if they're generally compacted in code with nesting and typedefs. Sean
Jul 30 2004
prev sibling parent "Walter" <newshound digitalmars.com> writes:
Ok, I see. I thought it was something else.
Jul 29 2004
prev sibling next sibling parent reply Nick <Nick_member pathlink.com> writes:
In article <cebp7v$1pt7$2 digitaldaemon.com>, Matthew says...

[...]
But one cannot do something such as

    int j = accumulate(l.begin(), l.end(), 0);

To do this, one has to provide the type(s), as in:

    int j = accumulate!(List!(int).iterator, int)(l.begin ...)

Just shooting out an idea here: Could you make accumulate() a (static?) member of List? Then you could get away with # int j = l.accumulate(l.begin(), l.end(), 0); since accumulate() would then already be instantiated as (List!(int).iterator, int). To avoid rewriting accumulate() and all the other algorithms you would of course have to put them in a base class or use mixins or something. Also, you can't add user defined algorithms this way. Nick
Jul 29 2004
parent "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Nick" <Nick_member pathlink.com> wrote in message
news:cec125$1tem$1 digitaldaemon.com...
 In article <cebp7v$1pt7$2 digitaldaemon.com>, Matthew says...

 [...]
But one cannot do something such as

    int j = accumulate(l.begin(), l.end(), 0);

To do this, one has to provide the type(s), as in:

    int j = accumulate!(List!(int).iterator, int)(l.begin ...)

Just shooting out an idea here: Could you make accumulate() a (static?) member of List? Then you could get away with # int j = l.accumulate(l.begin(), l.end(), 0); since accumulate() would then already be instantiated as (List!(int).iterator, int). To avoid rewriting accumulate() and all the other algorithms you would of course have to put them in a base class or use mixins or something. Also, you can't add user defined algorithms this way.

Short answer: yes. This is exactly what I'm doing with the transformation methods in DTL. But the problem in general is that there are an unbounded number of methods that one could seek to add. Whatever anyone might think about STL, its separation of containers from algorithms, communicating via iterators, is an incredibly elegant and powerful idea. With the DTL containers, I'm restricting myself to a small set of operations, currently called: - select() - returns a range containing elements matching a given function/delegate/functor predicate - reject() - the opposite of select() - collect() - returns a range containing the same number of elements, where each value in the returned collection is the result of applying a given function/delegate/functor transformation function and one or two others. Wrt things like accumulate(), I'd argue that one must be content to use foreach if we don't get implicit instantiaton.
Jul 29 2004
prev sibling next sibling parent reply Sha Chancellor <schancel pacific.net> writes:
In article <cebp7v$1pt7$2 digitaldaemon.com>,
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote:

 "Sha Chancellor" <schancel pacific.net> wrote in message 
 news:schancel-C6DC99.10061329072004 digitalmars.com...
 Matthew,

    Compiler bugs aside, you forté make a few silly arguments.  Such as,
 that you can't support iterators because you can't overload the *
 operator.  What kind of BS is that?

I said "Neither of these preclude the creation of iterators - and I've done a little of that - ". Maybe you should have read it more thoroughly. :-(

No, but you did say : "D cannot support generic programming." Presumably, you were griping about it. Also, you said "it means they're not really very usable." Please elaborate.
 Overloading operators for no apparent good reason is STLs forté.  Obj-C
 and Cocoa have lots of templates and it doesn't have any operator
 overloading what-so-ever.

 STLs and C++ use operator overloading incessantly  because somehow their
 designers thinks it looks cool.

Wrong. (Or half wrong anyway). They overload the operators for conformance with pointers. Surely you know/recognise that a pointer is a RandomAccess iterator?

An iterator class is still a class, not a pointer. End of story. Don't confuse syntax for types with needless philosophical semantics. And a pointer is not a RandomAccess iterator, although it might be a random access iterator....
 It's been my opinion - espoused often on the D ngs over the last year or so - 
 that we don't need to directly support
 pointers as "first class" members of the enumerable club in D, since D's 
 arrays (and any UDTs) are already enumerable
 via foreach().
 
 This has been one of the motivating factors in my design toward a 
 foreach-oriented library, with the consequences that
 two of the four of the enumeration "modes" / "models" in DTL are based on 
 foreach.

Where are you going with this?
 
   IT confuses whether or not an iterator
 is a pointer or an object.  Why not just have a real function call:

Pointers, as explained above.

Classes, as explained above.
 
 Iterator.getMyFuckingInstanceNow()

Yes. The iterators I've played with have taken this approach.

It makes the implementation more clear for novice programmers.
 
 No instead we must confuse things by going (*Iterator)

 Iterators in STL suck, plain and simple.  Most of C++'s standard
 template library is overly complicated and obtuse for the job.  Not to
 mention buggy between implementations.

At the risk of pissing off YAA, I think this is naive. (Aside: pretty much everyone in the world apart from fellow Englishmen tend to get shirty if I mention the "n" word. For my part, it's much less offensive to have one's opinion called naive, than ill-informed, or half-considered, or plain dumb. So naive it is.)

Which part of overly complicated, obtuse, and buggy between implementations is naive and ill-informed? It doesn't matter if you have a special method to tie my shoes if I don't know about it, or can't remember how to use it. Especially when I can tie my own shoes, I'll probably just go do that, rather than look up your auto-shoe-lace-tier.
 Yes, C++'s library is complex, and it's going to get a lot more so in 
 C++-0.x.
 
 Yes, templates are hard to understand, and template implementations are 
 virtually impossible not to become dialecticised
 (?). Consider, even though I value simplicity in templates more highly than 
 pretty much any other aspect, I've done some
 reasonably complex things with STLSoft, and some people have commented that 
 that stuff is pretty
 advanced/over-complicated/obtuse/cool/stupid. Point being, I understand the 
 complex techniques I've used in STLSoft, but
 I cannot understand the implementation of even small Boost modules, for the 
 same reason that some Boosters cannot
 understand STLSoft: C++ is too complex.

Boost is too complicated too.
 But, C++ is powerful, and that's why we stick with it. To stand on the 
 sidelines and just shout about how hard it is is
 plain silly, until something replaces it. I don't think I speak out of turn 
 to say that Walter would like that to be D.
 To achieve that D *must* support the currently unmatched power of C++. The 
 other issues are (i) whether it becomes as
 hard, and (ii) whether it engenders the confusing and disenfranchising 
 dialectisation (?) of C++. I my opinion we can
 live with (i) if we must, because most developers are reasonably intelligent. 
 But (ii) will kill it, because learning
 limitless dialects is incredibly irritating and a waste of effort. If D ends 
 up being as big a bag of wild-woman's
 knitting as C++, then what's the motivation to move to D?

As I would like to see D become also.
 [Apologies to any wild-women who have neat knitting.]
 
 As for implicit instantation,  Why is this required to make an iterator?

Sigh. Either I have all the didactic talent of your average post-graduate tutorial host, or you've not read my post correctly. What I said was that to use iterators generically, and without iterators sharing a common interface (e.g. IObjectEnumerator, or IEnumerator!(int)), we need implicit instantiation.

Explain why this would help please.
 
 Maybe I'm missing the boat, but what's wrong with something like this:

 class List(T) {
    alias ListIterator(T) iterator;

    ...
 }

 List!(int).iterator iter;
 // Hell you could probably even take your existing instance and do:
 // Foo.iterator iter;  In fact you should be able to but it seems you
 can't  This would allow you to also do aliases for stuff like:

 iter.BaseType t; and do your generic manipulation functions.
 or List.BaseType; etc.  And have aliases in your class.

This kind of thing is already done. One can (or at least could - I've changed a lot of things since April, when I last built and tested with -version=UseIterators) do things such as: List!(int) l = new ... for(List!(int).iterator b = l.begin(), e = l.end(); b != e; b.next()) { int i = b.value(); } But one cannot do something such as int j = accumulate(l.begin(), l.end(), 0); To do this, one has to provide the type(s), as in: int j = accumulate!(List!(int).iterator, int)(l.begin ...) [Actually, since all iterators must be classes, I've prescribed that they all have a standard set of member types, so it's possible to deduce the value type, as in: int j = accumulate!(List!(int).iterator)(l.begin ...) ] Given that the (currently) only useful mode for iterators is where the type is explicit, it's my opinion (and that of others, I think) that there's no benefit over foreach, hence the focus on foreach for the container enumeration, and the enumeration of transformed ranges, as in: foreach(int i; l) { . . . } and foreach(int i; l.select(IsOdd)) { . . . }
 I say we bitch at walter.

 for( iter = myList.begin();  iter != myList.end(); iter++ )
 {
    printf(iter.getMyFuckingInstanceNow().toString());
 }

 This is nearly identical to what C++ does, and i really don't see why it
 wouldn't work.

It can work. It does work. I've mentioned this many times on the NGs.
  You could probably leave the instance-getter function
 name a little shorter though.

D'ya think?
 Maybe you could expand as to why this sucks? (Besides that it's similar
 to STL which sucks)

Again, a pointless comment that suggests ignorance, or a fondness for glib statements at best. STL is a mostly marvellous idea/technology, but the bits of it that are bad are glaringly bad. It does not suck, it's the current best effort at powerful generic programming. Just because something has bad parts doesn't mean it sucks. If that's the case, then everything sucks. It's my hope and, I would think, that of most people in D, that DTL can be (nearly) as powerful as STL, but jettison most of the glaringly bad parts. I don't expect DTL to be without flaws, but it needs to be simpler, and less prone to spawning dialects, than STL. And it needs more power than the language currently provides.

STL is more overloaded and bulky than most things need.
 And as for your template example:


 #    template dump(T) {
 #    void dump(IContainer!(T) c)
 #    {
 #        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
 #        {
 #
 #        }
 #    }
 #    }
 #
 #    List!(int, IContainer!(int))           l = new ...;
 #
 #    dump(l);

 For one, making those aliases all over is just as ugly and hard to
 remember as just specifying it.  Second:

 You're passing in l of type List!(int, IContainer!(int)), and then
 proceed to do this with it:

 IContainer!(List!(int, IContainer!(int))) c.

 Is that really what you were intending to do?

I'm afraid I can't follow the last section. Can you rephrase?

Never mind, the way you used templates there is a bit odd. Let me see if I've got this correct: Since you've got a function with the same name as the template some magic happens here presumably. dump( List!(int, IContainer!(int)) ); Would be equivalent to: dump!(int).dump( List!(int, IContainer!(int)) ); in this case? And List!(int, IContainer!(int)) would be seen as a IContainer!(int) due to the specified interface?
 It seems like templates should support something along these lines:

 #   template dump( T : IContainer!(T) ) {
 #      void dump( IContainer!(t) ) {
 #      }
 #   }

 Although I seriously doubt that works as it is.

What's "t"? If you mean "T", then I presume the different you're providing is the constraint on T to be "in" an IContainer!(T)?? Doesn't the function signature already do that? Please elucidate.

I'm sorry, I made a capitalization error. Yes I would be constraining it to be in an IContainer, But also you would then deduce the template's type via template specialization. Then you could use the base type. But yes, it looks like your example already does that, I overlooked it.
Jul 29 2004
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Sha Chancellor" <schancel pacific.net> wrote in message
news:schancel-A163D2.18113529072004 digitalmars.com...
 In article <cebp7v$1pt7$2 digitaldaemon.com>,
  "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote:

 "Sha Chancellor" <schancel pacific.net> wrote in message
 news:schancel-C6DC99.10061329072004 digitalmars.com...
 Matthew,

    Compiler bugs aside, you forté make a few silly arguments.  Such as,
 that you can't support iterators because you can't overload the *
 operator.  What kind of BS is that?

I said "Neither of these preclude the creation of iterators - and I've done a little of that - ". Maybe you should have read it more thoroughly. :-(

No, but you did say : "D cannot support generic programming." Presumably, you were griping about it. Also, you said "it means they're not really very usable." Please elaborate.

They require explicit instantiation. That's not very usable.
 Overloading operators for no apparent good reason is STLs forté.  Obj-C
 and Cocoa have lots of templates and it doesn't have any operator
 overloading what-so-ever.

 STLs and C++ use operator overloading incessantly  because somehow their
 designers thinks it looks cool.

Wrong. (Or half wrong anyway). They overload the operators for conformance with pointers. Surely you know/recognise that a pointer is a RandomAccess iterator?

An iterator class is still a class, not a pointer. End of story. Don't confuse syntax for types with needless philosophical semantics.

I have no idea what this means. I haven't said a class is a pointer.
 And a pointer is not a RandomAccess iterator, although it might be a
 random access iterator....

Are you kidding here? I don't get the point of saying this, unless you're attempting to be funny.
 It's been my opinion - espoused often on the D ngs over the last year or so -
 that we don't need to directly support
 pointers as "first class" members of the enumerable club in D, since D's
 arrays (and any UDTs) are already enumerable
 via foreach().

 This has been one of the motivating factors in my design toward a
 foreach-oriented library, with the consequences that
 two of the four of the enumeration "modes" / "models" in DTL are based on
 foreach.

Where are you going with this?

No idea. You've cut out your part of the thread to which I was responding.
   IT confuses whether or not an iterator
 is a pointer or an object.  Why not just have a real function call:

Pointers, as explained above.

Classes, as explained above.

You were talking about STL, which is in C++, and which supports pointers. Are you suggesting that STL should not support pointers, or are you trying to rewrite the history of this thread to make it seem you were talking about D's iterators, which, as I've been saying for the best part of 12 months, are classes. Seems to me like you're just attempting to be smart/argumentative for the sake of it. Maybe it's me ...
 Iterator.getMyFuckingInstanceNow()

Yes. The iterators I've played with have taken this approach.

It makes the implementation more clear for novice programmers.

Agreed. (Thank god!)
 No instead we must confuse things by going (*Iterator)

 Iterators in STL suck, plain and simple.  Most of C++'s standard
 template library is overly complicated and obtuse for the job.  Not to
 mention buggy between implementations.

At the risk of pissing off YAA, I think this is naive. (Aside: pretty much everyone in the world apart from fellow Englishmen tend to get shirty if I mention the "n" word. For my part, it's much less offensive to have one's opinion called naive, than ill-informed, or half-considered, or plain dumb. So naive it is.)

Which part of overly complicated, obtuse, and buggy between implementations is naive and ill-informed?

You said "Iterators in STL suck, plain and simple". That is the part I did describe as naive, and which I now describe as ignorant. Let's hope we don't have another round and go for stupid.
 It doesn't matter if you
 have a special method to tie my shoes if I don't know about it, or can't
 remember how to use it.  Especially when I can tie my own shoes, I'll
 probably just go do that, rather than look up your auto-shoe-lace-tier.


 Yes, C++'s library is complex, and it's going to get a lot more so in
 C++-0.x.

 Yes, templates are hard to understand, and template implementations are
 virtually impossible not to become dialecticised
 (?). Consider, even though I value simplicity in templates more highly than
 pretty much any other aspect, I've done some
 reasonably complex things with STLSoft, and some people have commented that
 that stuff is pretty
 advanced/over-complicated/obtuse/cool/stupid. Point being, I understand the
 complex techniques I've used in STLSoft, but
 I cannot understand the implementation of even small Boost modules, for the
 same reason that some Boosters cannot
 understand STLSoft: C++ is too complex.

Boost is too complicated too.

Agreed. I would say too complex, but for the fact that a Booster might legitimately level the same at STLSoft
 But, C++ is powerful, and that's why we stick with it. To stand on the
 sidelines and just shout about how hard it is is
 plain silly, until something replaces it. I don't think I speak out of turn
 to say that Walter would like that to be D.
 To achieve that D *must* support the currently unmatched power of C++. The
 other issues are (i) whether it becomes as
 hard, and (ii) whether it engenders the confusing and disenfranchising
 dialectisation (?) of C++. I my opinion we can
 live with (i) if we must, because most developers are reasonably intelligent.
 But (ii) will kill it, because learning
 limitless dialects is incredibly irritating and a waste of effort. If D ends
 up being as big a bag of wild-woman's
 knitting as C++, then what's the motivation to move to D?

As I would like to see D become also.

Goodo
 [Apologies to any wild-women who have neat knitting.]

 As for implicit instantation,  Why is this required to make an iterator?

Sigh. Either I have all the didactic talent of your average post-graduate tutorial host, or you've not read my post correctly. What I said was that to use iterators generically, and without iterators sharing a common interface (e.g. IObjectEnumerator, or IEnumerator!(int)), we need implicit instantiation.

Explain why this would help please.

There's no implicit instantiation. So to use an iterator class with a "generic" algorithm either means that one uses a template algorithm and explicitly instantiates its type, or one uses a non-template algorithm that manipulates via an interface and passes only types that inherit from that interface to it. The DTL containers can all inherit from an otherwise defaulted second template parameter, which defines their "parent" interface, as in: Vector!(int, IObjectContainer) v; List!(int, IObjectContainer) l; Now v and l are both polymorphically related about the IObjectContainer interface, and either can be passed to a function that knows how to use an IObjectContainer. dump(IObjectContainer c); dump(v); dump(l); The self same thing would apply to iterators. We would define iterator interfaces, either Object-based (i.e. IObjectIterator), or parameterised (e.g. IIterator!(int), from which our containers' iterator classes would derive. Let's say we'll derive the iterator classes for List!(T) and Vector!(T) from IIterator!(T), thus: int accumulate(IIterator!(int) from, IIterator!(int) to, int initial); int r1 = accumulate(v.begin(), v.end(), 0); int r2 = accumulate(l.begin(), l.end(), 0); But this is not generic programming, or at least not as I understand the term. accumulate() is not a generic algorith, it is a specifi function, whose non-template parameters are the type IIterator!(int). To accumulate BigInts, or doubles, we'd need another implementation of accumulate() for each. Of course, one could take the next step, and get almost all the way there, template accumulate(T) { T accumulate(IIterator!(T) from, IIterator!(T) to, T initial) { . . . accumulation ... }} Vector!(BigInt, IObjectContainer) v; List!(double, IObjectContainer) l; BigInt r1 = accumulate!(BigInt)(v.begin(), v.end(), new BigInt(0)); double r2 = accumulate!(double)(v.begin(), v.end(), 0.0); But this is still not generic programming. There are several problems: 1. One must specify the type in the explicit parameterisation of accumulate(). This can be considered a minor problem in most cases 2. The processing is necessarily inefficient, because it's using virtual functions to access the elements, via the interface 3. The definition and use of accumulate!() is effectively tied to IIterator!(). Try incorporating another accumulate!() based on another interface into the mix! 4. There is no mechanism - or at least I've not thought of one - for short-circuiting operations based on the iterator refinement. In other words, how do we get a random access iterator class to do an efficient distance()? (Actually, I'm having some ideas now ... but it's likely to be nasty stuff.)
 Maybe I'm missing the boat, but what's wrong with something like this:

 class List(T) {
    alias ListIterator(T) iterator;

    ...
 }

 List!(int).iterator iter;
 // Hell you could probably even take your existing instance and do:
 // Foo.iterator iter;  In fact you should be able to but it seems you
 can't  This would allow you to also do aliases for stuff like:

 iter.BaseType t; and do your generic manipulation functions.
 or List.BaseType; etc.  And have aliases in your class.

This kind of thing is already done. One can (or at least could - I've changed a lot of things since April, when I last built and tested with -version=UseIterators) do things such as: List!(int) l = new ... for(List!(int).iterator b = l.begin(), e = l.end(); b != e; b.next()) { int i = b.value(); } But one cannot do something such as int j = accumulate(l.begin(), l.end(), 0); To do this, one has to provide the type(s), as in: int j = accumulate!(List!(int).iterator, int)(l.begin ...) [Actually, since all iterators must be classes, I've prescribed that they all have a standard set of member types, so it's possible to deduce the value type, as in: int j = accumulate!(List!(int).iterator)(l.begin ...) ] Given that the (currently) only useful mode for iterators is where the type is explicit, it's my opinion (and that of others, I think) that there's no benefit over foreach, hence the focus on foreach for the container enumeration, and the enumeration of transformed ranges, as in: foreach(int i; l) { . . . } and foreach(int i; l.select(IsOdd)) { . . . }
 I say we bitch at walter.

 for( iter = myList.begin();  iter != myList.end(); iter++ )
 {
    printf(iter.getMyFuckingInstanceNow().toString());
 }

 This is nearly identical to what C++ does, and i really don't see why it
 wouldn't work.

It can work. It does work. I've mentioned this many times on the NGs.
  You could probably leave the instance-getter function
 name a little shorter though.

D'ya think?
 Maybe you could expand as to why this sucks? (Besides that it's similar
 to STL which sucks)

Again, a pointless comment that suggests ignorance, or a fondness for glib statements at best. STL is a mostly marvellous idea/technology, but the bits of it that are bad are glaringly bad. It does not suck, it's the current best effort at powerful generic programming. Just because something has bad parts doesn't mean it sucks. If that's the case, then everything sucks. It's my hope and, I would think, that of most people in D, that DTL can be (nearly) as powerful as STL, but jettison most of the glaringly bad parts. I don't expect DTL to be without flaws, but it needs to be simpler, and less prone to spawning dialects, than STL. And it needs more power than the language currently provides.

STL is more overloaded and bulky than most things need.

Agreed.
 And as for your template example:


 #    template dump(T) {
 #    void dump(IContainer!(T) c)
 #    {
 #        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
 #        {
 #
 #        }
 #    }
 #    }
 #
 #    List!(int, IContainer!(int))           l = new ...;
 #
 #    dump(l);

 For one, making those aliases all over is just as ugly and hard to
 remember as just specifying it.  Second:

 You're passing in l of type List!(int, IContainer!(int)), and then
 proceed to do this with it:

 IContainer!(List!(int, IContainer!(int))) c.

 Is that really what you were intending to do?

I'm afraid I can't follow the last section. Can you rephrase?

Never mind, the way you used templates there is a bit odd. Let me see if I've got this correct: Since you've got a function with the same name as the template some magic happens here presumably. dump( List!(int, IContainer!(int)) ); Would be equivalent to: dump!(int).dump( List!(int, IContainer!(int)) ); in this case? And List!(int, IContainer!(int)) would be seen as a IContainer!(int) due to the specified interface?

Walter has chosen to allow (in fact it's mandate, which is an irritation all its own) templates with only one member, whose name is the same as the template, to be specified without the qualification. i.e. one specifies List!(int), rather than List!(int).List.
 It seems like templates should support something along these lines:

 #   template dump( T : IContainer!(T) ) {
 #      void dump( IContainer!(t) ) {
 #      }
 #   }

 Although I seriously doubt that works as it is.

What's "t"? If you mean "T", then I presume the different you're providing is the constraint on T to be "in" an IContainer!(T)?? Doesn't the function signature already do that? Please elucidate.

I'm sorry, I made a capitalization error.

No worries. That's what I figured.
 Yes I would be constraining it to be in an IContainer, But also you
 would then deduce the template's type via template specialization.  Then
 you could use the base type.  But yes, it looks like your example
 already does that, I overlooked it.

No worries.
Jul 29 2004
parent Sha Chancellor <schancel pacific.net> writes:
In article <cec9sf$20qk$1 digitaldaemon.com>,
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote:

 There's no implicit instantiation. So to use an iterator class with a 
 "generic" algorithm either means that one uses a
 template algorithm and explicitly instantiates its type, or one uses a 
 non-template algorithm that manipulates via an
 interface and passes only types that inherit from that interface to it.
 
 The DTL containers can all inherit from an otherwise defaulted second 
 template parameter, which defines their "parent"
 interface, as in:
 
     Vector!(int, IObjectContainer)    v;
     List!(int, IObjectContainer)        l;
 
 Now v and l are both polymorphically related about the IObjectContainer 
 interface, and either can be passed to a
 function that knows how to use an IObjectContainer.
 
     dump(IObjectContainer c);
 
     dump(v);
     dump(l);
 
 The self same thing would apply to iterators. We would define iterator 
 interfaces, either Object-based (i.e.
 IObjectIterator), or parameterised (e.g. IIterator!(int), from which our 
 containers' iterator classes would derive.
 Let's say we'll derive the iterator classes for List!(T) and Vector!(T) from 
 IIterator!(T), thus:
 
     int accumulate(IIterator!(int) from, IIterator!(int) to, int initial);
 
     int     r1    =    accumulate(v.begin(), v.end(), 0);
     int     r2    =    accumulate(l.begin(), l.end(), 0);
 
 But this is not generic programming, or at least not as I understand the 
 term. accumulate() is not a generic algorith,
 it is a specifi function, whose non-template parameters are the type 
 IIterator!(int). To accumulate BigInts, or doubles,
 we'd need another implementation of accumulate() for each.
 
 Of course, one could take the next step, and get almost all the way there,
 
     template accumulate(T) { T accumulate(IIterator!(T) from, IIterator!(T) 
     to, T initial)
     {
         . . . accumulation ...
     }}
 
     Vector!(BigInt, IObjectContainer)    v;
     List!(double, IObjectContainer)        l;
 
     BigInt      r1    =    accumulate!(BigInt)(v.begin(), v.end(), new 
     BigInt(0));
     double     r2    =    accumulate!(double)(v.begin(), v.end(), 0.0);
 
 But this is still not generic programming. There are several problems:

 1. One must specify the type in the explicit parameterisation of 
 accumulate(). This can be considered a minor problem in
 most cases

It's a bit of extra typing, I'll give you that. But with the aliases for instances bug fixed. You could presumably do: accumulate!(v.baseType)(v.begin(), v.end(), 0.0) now.
 2. The processing is necessarily inefficient, because it's using virtual 
 functions to access the elements, via the
 interface

Granted. However, if D would allow you to specify greater restrictions on templates, as was proposed in this thread, (and walter liked) then the accumulate function could accept not a Iterator!(T) from. But rather a T, thus allowing it to be non-virtual. Presumably just requiring T to be a child of a IObjectContainer. Would that still cause the methods to be virtual though, just because they're conforming to an interface? Probably would, silly me. Could you give an example of how the accumulate function would be defined in C++?
 3. The definition and use of accumulate!() is effectively tied to 
 IIterator!(). Try incorporating another accumulate!()
 based on another interface into the mix!

D should find the most specialized case of accumulate. If it doesn't, this needs to be fixed.
 4. There is no mechanism - or at least I've not thought of one - for 
 short-circuiting operations based on the iterator
 refinement. In other words, how do we get a random access iterator class to 
 do an efficient distance()? (Actually, I'm
 having some ideas now ... but it's likely to be nasty stuff.)

Not sure what you mean here. I presume you mean a templated distance specialized for a RandomAccessIterator? Shouldn't that work if D finds the most specialized case, rather than the first one it finds?
Jul 30 2004
prev sibling next sibling parent reply Nick <Nick_member pathlink.com> writes:
In article <cebp7v$1pt7$2 digitaldaemon.com>, Matthew says...

[Actually, since all iterators must be classes, I've prescribed that they all
have a standard set of member types, so
it's possible to deduce the value type, as in:

    int j = accumulate!(List!(int).iterator)(l.begin ...)
]

Actually, you could do it this way even with pointers: # template accumulate(T: T*) { /* pointer iterators go here */ } # template accumulate(T: IteratorBase) { /* class iterators here */ } but it would mean you'd have to implement everything twice, which isn't exactly optimal :)
Given that the (currently) only useful mode for iterators is where the type
 is explicit, it's my opinion (and that of others, I think) that there's no
 benefit over foreach, hence the focus on foreach for the container
 enumeration,

I hope this doesn't mean you want to drop algorithms from DTL altogether? I think replacing sort() with a foreach will be a tad difficult. Nick
Jul 29 2004
parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Nick" <Nick_member pathlink.com> wrote in message
news:cec8k4$209b$1 digitaldaemon.com...
 In article <cebp7v$1pt7$2 digitaldaemon.com>, Matthew says...

[Actually, since all iterators must be classes, I've prescribed that they all
have a standard set of member types, so
it's possible to deduce the value type, as in:

    int j = accumulate!(List!(int).iterator)(l.begin ...)
]

Actually, you could do it this way even with pointers: # template accumulate(T: T*) { /* pointer iterators go here */ } # template accumulate(T: IteratorBase) { /* class iterators here */ } but it would mean you'd have to implement everything twice, which isn't exactly optimal :)
Given that the (currently) only useful mode for iterators is where the type
 is explicit, it's my opinion (and that of others, I think) that there's no
 benefit over foreach, hence the focus on foreach for the container
 enumeration,

I hope this doesn't mean you want to drop algorithms from DTL altogether? I think replacing sort() with a foreach will be a tad difficult.

I think we will end up with some form of iterator-based algorithm support, but I am with Walter in thinking that the "D" way will not revolve around iterators. Rather it will be (possibly transformed and filtered) foreach-able entities. Notwithstanding that, it's my intent all along that DTL will also support Java-style enumeration and STL-style enumeration. I don't see why anyone should be forced to go one way, when we can support them all (without efficiency costs of features not being used, of course).
Jul 30 2004
parent Juanjo =?ISO-8859-15?Q?=C1lvarez?= <juanjuxNO SPAMyahoo.es> writes:
Matthew wrote:

 Notwithstanding that, it's my intent all along that DTL will also support
 Java-style enumeration and STL-style enumeration. I don't see why anyone
 should be forced to go one way, when we can support them all (without
 efficiency costs of features not being used, of course).

That's exactly what Troll Tech is doing with their (still unreleased) Qt 4.0. I also like having a choice.
Jul 30 2004
prev sibling parent Matthis Becker <Matthis_member pathlink.com> writes:
 To achieve that D *must* support the currently unmatched power of C++. 

Hmm, isn't LIPS much more powerfull?
Aug 01 2004
prev sibling next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Matthew" <admin.hat stlsoft.dot.org> wrote in message
news:ceaeid$18ir$1 digitaldaemon.com...
 I'm almost at the end of my tether here. There are so many problems with

 seriously whether it's going to be possible to have a decent generic

I think that's to be expected with a new generic language design. After all, the creator of STL (Stepanov?) went through multiple design iterations with Stroustrup. None of us are smart enough to get it all right the first, second, or even third time. You do have a tendency to push the envelope past its limits (I see that in your C++ code!), and that's good for finding what those limits and weaknesses are. Generic programming in D is also going to be in a different style than in C++, and use different techniques. We have to carefully think about whether the problems are bugs in the compiler, bugs in the language design, or bugs in our thinking - perhaps we haven't discovered the right D way to do generic programming. One thing I am pretty sure of, the C++ way of doing STL isn't the right way for D. I personally find STL to be impenetrably obtuse. I refuse to believe that is the best way of doing generics. I think it's great that you are taking a different approach than STL does, and are open to inventive new ways of thinking about it.
 Alas, this crashes the compiler. Sigh ...

That's my problem, send me the test case!
 4. Iterator based approach. There are two reasons why STL-iterators are

 instantiation, and (ii) there is no operator *(). Neither of these

 little of that - including adaptors for arrays/pointers, but it means

Not having implicit instantiation can make the code a bit more verbose, but it shouldn't be that bad. As for operator*(), why should that be a problem? Just define a member function called, I'm sure there's a better name, operatorStar(), and instead of *foo, type foo.operatorStar().
 1. Beyond a significant, but indefinable level of complexity, the

 symbols, while chopping out any *unrelated* parts makes them locatable

 Nonetheless, that's a bug in the compiler/linker, and doesn't get my blood

 and at this stage a little bugginess is not the crime of the century.

If you can't reduce it, you can't reduce it. Send a reproducible example.
 2. Templates in D are instantiated en masse. What "en masse" actually

 true rules regarding instantation.

The true rule is the whole template is compiled when it is instantiated. (This is unlike C++, which does "lazy instantiation", which means it only compiles the bits of a template that are actually used.)
 But what I can say is that having a template such as the following is

 f**cked:

     template TransformedRange(R, F, T) { class TransformedRange
      : public NotionalRange!(T)
     {
       . . .
         this(R r, F f)
         {
             m_r = r.dup;
             m_f = f;
         }
         this(R r)
         {
             m_r = r.dup;
             m_f = new filter_type();
         }

 Without worrying too much about the details of what a TransformedRange

 not have a default constructor, one cannot instantiate TransformedRange

 *is never called*!

I don't get it from the example given.
 3. There's no implicit instantiation. This has *massive* consequences,

 - We can't have iterators (as noted above). This is fine for reading from

 "generically" insert into container ranges in the same (or any analogous)

Can you give a canonical example?
 - In order to support the parameterisable interface (e.g.

 way to manipulate built-in types, objects and structs. For some things,

 functions. Alas, there seems to be no way to discriminate structs via

 containers do not support polymorphic interfaces when storing structs.

I hadn't realized that structs and classes cannot be discriminated in template specialization, you're right. How does the following look: template foo(T : struct) and: template foo(T : class) ?
 4. D's import stuff just blows my mind! As a related issue to the boxing

 conflicts between the toString() functions in std.string and my box and

 private imports, to no avail. I concede that this might be my fault, and I

 rules, but as it currently looks to me, it looks bad and stupid.

I suspect the problem you're running into is that there's a toString in std.string, and a toString in foo. Then, in bar.d: import std.string; import foo; ... x = toString(y); ... This will bring up a conflict, because which toString do you want, the one in std.string or the one in foo? The compiler doesn't know. The rule is that name lookup happens first, *then* overload matching. If you want to overload them together: import std.string; import foo; alias std.string.toString toString; alias foo.toString toString; ... x = toString(y); ... now there's no longer a conflict, because toString is found in the current module (the aliases), and the aliases are chased down for overload resolution. It is this way because it gives you complete control over overloading.
 So, what do we do about this? Walter's stock response is to boil it down,

 non-trivial projects such as DTL, or I'm simply not smart enough to do it.

 of a too-complex design, but I think that's crap: the concept is simple,

 (Very similar techniques that I've experimented on in C++ work fine.)

If you can't do it, you can't do it. Send what you can do.
 Rather than having things boiled down, I think the compiler should be

 information, so we can email a dump of that to Walter, and which will be

Trust me, sending me dumps would be quite useless.
Jul 29 2004
next sibling parent reply "C. Sauls" <ibisbasenji yahoo.com> writes:
Walter wrote:
 I hadn't realized that structs and classes cannot be discriminated in
 template specialization, you're right. How does the following look:
     template foo(T : struct)
 and:
     template foo(T : class)
 ?

Looks promising to me... how likely would the following be, as well? template foo(T : signed) template foo(T : unsigned) template foo(T : enum) I figure the more ways to narrow down the parameters, the better. Maybe templates could be further revised to have complex filters, such as: template foo(T : struct || (signed && enum)) This would take as parameter wither a struct, or an enum that derived from a signed type (byte,short,int,long,cent..). It would reject anything else, including an /un/signed enum (enum : ubyte,ushort,uint,ulong.ucent..). The use of conditional operators is arbitrary, I guess anything could be used... maybe even new 'or' and 'and' keywords that are template-only. But I'm sure a certain amount of this is dreaming on my part. :) -Chris S. -Invironz
Jul 29 2004
next sibling parent Stephan Wienczny <Stephan Wienczny.de> writes:
I like your idea.
Theres only a tiny drawback. For the compiler it's harder to find the 
right template. That should be something big W can handle *g*

Stephan

C. Sauls wrote:
 
 
 Looks promising to me... how likely would the following be, as well?
     template foo(T : signed)
     template foo(T : unsigned)
     template foo(T : enum)
 
 I figure the more ways to narrow down the parameters, the better.  Maybe 
 templates could be further revised to have complex filters, such as:
     template foo(T : struct || (signed && enum))
 
 This would take as parameter wither a struct, or an enum that derived 
 from a signed type (byte,short,int,long,cent..).  It would reject 
 anything else, including an /un/signed enum (enum : 
 ubyte,ushort,uint,ulong.ucent..).  The use of conditional operators is 
 arbitrary, I guess anything could be used... maybe even new 'or' and 
 'and' keywords that are template-only.
 
 But I'm sure a certain amount of this is dreaming on my part.  :)
 
 -Chris S.
 -Invironz

Jul 29 2004
prev sibling next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"C. Sauls" <ibisbasenji yahoo.com> wrote in message
news:cebjlk$1n2q$1 digitaldaemon.com...
 Walter wrote:
 I hadn't realized that structs and classes cannot be discriminated in
 template specialization, you're right. How does the following look:
     template foo(T : struct)
 and:
     template foo(T : class)
 ?

Looks promising to me... how likely would the following be, as well? template foo(T : signed) template foo(T : unsigned) template foo(T : enum) I figure the more ways to narrow down the parameters, the better. Maybe templates could be further revised to have complex filters, such as: template foo(T : struct || (signed && enum))

It's a good idea. I've been thinking something along those lines for 2.0.
Jul 29 2004
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Walter" <newshound digitalmars.com> wrote in message
news:cebm6f$1o79$1 digitaldaemon.com...
 "C. Sauls" <ibisbasenji yahoo.com> wrote in message
 news:cebjlk$1n2q$1 digitaldaemon.com...
 Walter wrote:
 I hadn't realized that structs and classes cannot be discriminated in
 template specialization, you're right. How does the following look:
     template foo(T : struct)
 and:
     template foo(T : class)
 ?

Looks promising to me... how likely would the following be, as well? template foo(T : signed) template foo(T : unsigned) template foo(T : enum) I figure the more ways to narrow down the parameters, the better. Maybe templates could be further revised to have complex filters, such as: template foo(T : struct || (signed && enum))

It's a good idea. I've been thinking something along those lines for 2.0.

Think 1.0!
Jul 29 2004
parent Gold Dragon <dragonwing dragonu.net> writes:
Matthew wrote:

 "Walter" <newshound digitalmars.com> wrote in message
news:cebm6f$1o79$1 digitaldaemon.com...
 
"C. Sauls" <ibisbasenji yahoo.com> wrote in message
news:cebjlk$1n2q$1 digitaldaemon.com...

Walter wrote:

I hadn't realized that structs and classes cannot be discriminated in
template specialization, you're right. How does the following look:
    template foo(T : struct)
and:
    template foo(T : class)
?

Looks promising to me... how likely would the following be, as well? template foo(T : signed) template foo(T : unsigned) template foo(T : enum) I figure the more ways to narrow down the parameters, the better. Maybe templates could be further revised to have complex filters, such as: template foo(T : struct || (signed && enum))

It's a good idea. I've been thinking something along those lines for 2.0.

Think 1.0!

novice programmer I wish I could but then again I'm not building something as complicated as a compiler that supports templates. This is a feature I was hoping to see also but I guess that if it wasn't in the docs then it probably won't be supported anytime soon. I just hope 2.0 doesn't take as long as 1.0 because that would suck. Five years just to see better template controls, well, maybe it won't take that long if Walter keeps releasing builds.
Jul 30 2004
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <cebjlk$1n2q$1 digitaldaemon.com>, C. Sauls says...
Looks promising to me... how likely would the following be, as well?
	template foo(T : signed)
	template foo(T : unsigned)
	template foo(T : enum)

I figure the more ways to narrow down the parameters, the better.  Maybe 
templates could be further revised to have complex filters, such as:
	template foo(T : struct || (signed && enum))

This would be fantastic. It would single-handedly eliminate a large bulk of meta garbage classes. Sean
Jul 29 2004
parent reply Juanjo =?ISO-8859-15?Q?=C1lvarez?= <juanjuxNO SPAMyahoo.es> writes:
Sean Kelly wrote:

 In article <cebjlk$1n2q$1 digitaldaemon.com>, C. Sauls says...
Looks promising to me... how likely would the following be, as well?
template foo(T : signed)
template foo(T : unsigned)
template foo(T : enum)

I figure the more ways to narrow down the parameters, the better.  Maybe
templates could be further revised to have complex filters, such as:
template foo(T : struct || (signed && enum))

This would be fantastic. It would single-handedly eliminate a large bulk of meta garbage classes.

Another strange idea, extending it to classes: template foo(T : ForwardIterable, BackwardIterable, Addable) Of couse ForwardIterable, BackwardIterable and Addable would be interfaces that the foo parameter must implement.
Jul 29 2004
parent reply Juanjo =?ISO-8859-15?Q?=C1lvarez?= <juanjuxNO SPAMyahoo.es> writes:
Juanjo Álvarez wrote:

 Sean Kelly wrote:
 
 In article <cebjlk$1n2q$1 digitaldaemon.com>, C. Sauls says...
Looks promising to me... how likely would the following be, as well?
template foo(T : signed)
template foo(T : unsigned)
template foo(T : enum)

I figure the more ways to narrow down the parameters, the better.  Maybe
templates could be further revised to have complex filters, such as:
template foo(T : struct || (signed && enum))

This would be fantastic. It would single-handedly eliminate a large bulk of meta garbage classes.

Another strange idea, extending it to classes: template foo(T : ForwardIterable, BackwardIterable, Addable) Of couse ForwardIterable, BackwardIterable and Addable would be interfaces that the foo parameter must implement.

And of course that syntax would not work because it takes the second and third interfaces as new parameters :). Anyway I've re-readed a little of the spec to find that you can do: interface ForwardIterable { void fwiterate(); } interface BackwardIterable { void bwiterate(); } interface Addable { /*opAdd and stuff */ } //Ugly iterface FwBwIterableAddable : ForwardIterable, BackwardIterable, Addable {} class A : FwBwIterableAddable { void fwiterate() { return 0;} void bwiterable() { return 0;} } template TFoo( T : FwBwIterableAddable ) {} main() { alias TFoo!(A)x; } But as you can see you have to declare an interface which inherits from all the interfaces and then declare the class to inherit from that interface to be allowed to limit the template parameter. Same effect, but uglier than just specifying a list of interfaces to the template parameter. Anyway I've done little D and none of that was very generic so I could be being really naive here (not that I care, that's one of my features ;).
Jul 29 2004
next sibling parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Juanjo Álvarez" <juanjuxNO SPAMyahoo.es> wrote in message
news:cec023$1svv$1 digitaldaemon.com...
 Juanjo Álvarez wrote:

 Sean Kelly wrote:

 In article <cebjlk$1n2q$1 digitaldaemon.com>, C. Sauls says...
Looks promising to me... how likely would the following be, as well?
template foo(T : signed)
template foo(T : unsigned)
template foo(T : enum)

I figure the more ways to narrow down the parameters, the better.  Maybe
templates could be further revised to have complex filters, such as:
template foo(T : struct || (signed && enum))

This would be fantastic. It would single-handedly eliminate a large bulk of meta garbage classes.

Another strange idea, extending it to classes: template foo(T : ForwardIterable, BackwardIterable, Addable) Of couse ForwardIterable, BackwardIterable and Addable would be interfaces that the foo parameter must implement.

And of course that syntax would not work because it takes the second and third interfaces as new parameters :). Anyway I've re-readed a little of the spec to find that you can do: interface ForwardIterable { void fwiterate(); } interface BackwardIterable { void bwiterate(); } interface Addable { /*opAdd and stuff */ } //Ugly iterface FwBwIterableAddable : ForwardIterable, BackwardIterable, Addable {} class A : FwBwIterableAddable { void fwiterate() { return 0;} void bwiterable() { return 0;} } template TFoo( T : FwBwIterableAddable ) {} main() { alias TFoo!(A)x; } But as you can see you have to declare an interface which inherits from all the interfaces and then declare the class to inherit from that interface to be allowed to limit the template parameter. Same effect, but uglier than just specifying a list of interfaces to the template parameter. Anyway I've done little D and none of that was very generic so I could be being really naive here (not that I care, that's one of my features ;).

I think we should just give Walter our requirements and expectations, and let his jogging brain come up with one of his customary neat minimalistic solutions.
Jul 29 2004
parent Juanjo =?ISO-8859-15?Q?=C1lvarez?= <juanjuxNO SPAMyahoo.es> writes:
Matthew wrote:
 
 I think we should just give Walter our requirements and expectations, and
 let his jogging brain come up with one of his customary neat minimalistic
 solutions.

What about adding a (categorized and prioritized) wishlist page on the wiki?
Jul 29 2004
prev sibling parent reply "C. Sauls" <ibisbasenji yahoo.com> writes:
 And of course that syntax would not work because it takes the second and
 third interfaces as new parameters :).

And then comes the joy of my suggested way of doing it... it opens up this possibility: template foo (T : Interface1 && Interface2 && Interface3) -or- template foo (T : Interface1 and Interface2 and Interface3) Although after looking at it I started wondering about my (signed && enum) trick... I don't think it would be good, at least not that way. I get stuck wondering how to interpret something like (signed && struct)... What would make a /struct/ "signed" or "unsigned"?? So it could lead to ambiguity, and nobody likes ambiguity... okay well some of the time. -Chris S. -Invironz
Jul 29 2004
parent reply Andy Friesen <andy ikagames.com> writes:
C. Sauls wrote:

 And of course that syntax would not work because it takes the second and
 third interfaces as new parameters :).

And then comes the joy of my suggested way of doing it... it opens up this possibility: template foo (T : Interface1 && Interface2 && Interface3) -or- template foo (T : Interface1 and Interface2 and Interface3) Although after looking at it I started wondering about my (signed && enum) trick... I don't think it would be good, at least not that way. I get stuck wondering how to interpret something like (signed && struct)... What would make a /struct/ "signed" or "unsigned"?? So it could lead to ambiguity, and nobody likes ambiguity... okay well some of the time.

Say we decide on some operations that can be done on types: T1 == T2 - T1 and T2 are the same type T1 > T2 - T1 is a subclass of T2? (iffy. T1 is a superset of T2) T1 >= T2 - subclass or equal D in T - declaration D is a member of instances of T T : TE - T is deduced from type expression TE Then we can borrow a little from C#, a little from ML, and do something like this: template ComplexTemplateThing(A, B, C, D) where A > Stream && // A must extend stream B > struct && // B "extends" struct void opApply(inout char[]) in C && // C instances must have an opApply method D : null[null] // specialization: 'null' matches all types. // (ML uses _ the same way) // Type D can be any associative array || // or D : C[] // permit D to be an array of C { ... } The only problem is that we've started turning into Lisp all over again. :) -- andy
Jul 29 2004
parent "Bent Rasmussen" <exo bent-rasmussen.info> writes:
Interesting... As long as we're dreaming* up stuff, here's a few more
ideas...

T map(S,T)(S x)
{
    ...
}

=>

template map(S,T)
{
    T map(S x)
    {
        ...
    }
}

(and then something a bit more far out...)

If "name: type" order was reversed (*)

map(S,T)(x: S): T

If there were inline type parameters

map (x: $S): $T

and perhaps

map (x: $S: {A, B} = A): $T

Explicit instantiation, e.g. map!(int,bit) -- $S = int, $T = bit.
I like that its concise, but I'm not too sure about its viability.

"Andy Friesen" <andy ikagames.com> wrote in message
news:cecgd6$231k$1 digitaldaemon.com...
 C. Sauls wrote:

 And of course that syntax would not work because it takes the second



 third interfaces as new parameters :).

And then comes the joy of my suggested way of doing it... it opens up this possibility: template foo (T : Interface1 && Interface2 && Interface3) -or- template foo (T : Interface1 and Interface2 and Interface3) Although after looking at it I started wondering about my (signed && enum) trick... I don't think it would be good, at least not that way. I get stuck wondering how to interpret something like (signed && struct)... What would make a /struct/ "signed" or "unsigned"?? So it could lead to ambiguity, and nobody likes ambiguity... okay well some of the time.

Say we decide on some operations that can be done on types: T1 == T2 - T1 and T2 are the same type T1 > T2 - T1 is a subclass of T2? (iffy. T1 is a superset of T2) T1 >= T2 - subclass or equal D in T - declaration D is a member of instances of T T : TE - T is deduced from type expression TE Then we can borrow a little from C#, a little from ML, and do something like this: template ComplexTemplateThing(A, B, C, D) where A > Stream && // A must extend stream B > struct && // B "extends" struct void opApply(inout char[]) in C && // C instances must have an opApply method D : null[null] // specialization: 'null' matches all types. // (ML uses _ the same way) // Type D can be any associative array || // or D : C[] // permit D to be an array of C { ... } The only problem is that we've started turning into Lisp all over again.

   -- andy

Jul 30 2004
prev sibling next sibling parent Sha Chancellor <schancel pacific.net> writes:
In article <cebjlk$1n2q$1 digitaldaemon.com>,
 "C. Sauls" <ibisbasenji yahoo.com> wrote:

 Walter wrote:
 I hadn't realized that structs and classes cannot be discriminated in
 template specialization, you're right. How does the following look:
     template foo(T : struct)
 and:
     template foo(T : class)
 ?

Looks promising to me... how likely would the following be, as well? template foo(T : signed) template foo(T : unsigned) template foo(T : enum) I figure the more ways to narrow down the parameters, the better. Maybe templates could be further revised to have complex filters, such as: template foo(T : struct || (signed && enum)) This would take as parameter wither a struct, or an enum that derived from a signed type (byte,short,int,long,cent..). It would reject anything else, including an /un/signed enum (enum : ubyte,ushort,uint,ulong.ucent..). The use of conditional operators is arbitrary, I guess anything could be used... maybe even new 'or' and 'and' keywords that are template-only. But I'm sure a certain amount of this is dreaming on my part. :) -Chris S. -Invironz

I really like the idea of complex filters on template specialization. Count my vote for.
Jul 29 2004
prev sibling parent reply Regan Heath <regan netwin.co.nz> writes:
I think in addition to simple properties like:
   template foo(T : struct)

A whole new thing should be used to specify several, like an interface but 
not, i.e.

requirements Bob {
   interface Interface1,..
   typeof ParentClass,..
   type unsigned,..
}

template foo(T : Bob)

so Bob contains declarations about the type eg, it's type, what interfaces 
it must support, even what it's derived from. all optional of course.

I am not suggesting a syntax. Simply that we might want to group 
requirements, give them a name and re-use them all over the place.

Regan

On Thu, 29 Jul 2004 14:37:15 -0500, C. Sauls <ibisbasenji yahoo.com> wrote:
 Walter wrote:
 I hadn't realized that structs and classes cannot be discriminated in
 template specialization, you're right. How does the following look:
     template foo(T : struct)
 and:
     template foo(T : class)
 ?

Looks promising to me... how likely would the following be, as well? template foo(T : signed) template foo(T : unsigned) template foo(T : enum) I figure the more ways to narrow down the parameters, the better. Maybe templates could be further revised to have complex filters, such as: template foo(T : struct || (signed && enum)) This would take as parameter wither a struct, or an enum that derived from a signed type (byte,short,int,long,cent..). It would reject anything else, including an /un/signed enum (enum : ubyte,ushort,uint,ulong.ucent..). The use of conditional operators is arbitrary, I guess anything could be used... maybe even new 'or' and 'and' keywords that are template-only. But I'm sure a certain amount of this is dreaming on my part. :) -Chris S. -Invironz

-- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 29 2004
next sibling parent resistor AT mac DOT com <resistor_member pathlink.com> writes:
Wow.  That's an awesome idea.  I second it!!

-Owen

In article <opsbxg00bk5a2sq9 digitalmars.com>, Regan Heath says...
I think in addition to simple properties like:
   template foo(T : struct)

A whole new thing should be used to specify several, like an interface but 
not, i.e.

requirements Bob {
   interface Interface1,..
   typeof ParentClass,..
   type unsigned,..
}

template foo(T : Bob)

so Bob contains declarations about the type eg, it's type, what interfaces 
it must support, even what it's derived from. all optional of course.

I am not suggesting a syntax. Simply that we might want to group 
requirements, give them a name and re-use them all over the place.

Regan

On Thu, 29 Jul 2004 14:37:15 -0500, C. Sauls <ibisbasenji yahoo.com> wrote:
 Walter wrote:
 I hadn't realized that structs and classes cannot be discriminated in
 template specialization, you're right. How does the following look:
     template foo(T : struct)
 and:
     template foo(T : class)
 ?

Looks promising to me... how likely would the following be, as well? template foo(T : signed) template foo(T : unsigned) template foo(T : enum) I figure the more ways to narrow down the parameters, the better. Maybe templates could be further revised to have complex filters, such as: template foo(T : struct || (signed && enum)) This would take as parameter wither a struct, or an enum that derived from a signed type (byte,short,int,long,cent..). It would reject anything else, including an /un/signed enum (enum : ubyte,ushort,uint,ulong.ucent..). The use of conditional operators is arbitrary, I guess anything could be used... maybe even new 'or' and 'and' keywords that are template-only. But I'm sure a certain amount of this is dreaming on my part. :) -Chris S. -Invironz

-- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

Jul 29 2004
prev sibling parent reply Juanjo =?ISO-8859-15?Q?=C1lvarez?= <juanjuxNO SPAMyahoo.es> writes:
Regan Heath wrote:

 I think in addition to simple properties like:
    template foo(T : struct)
 
 A whole new thing should be used to specify several, like an interface but
 not, i.e.
 
 requirements Bob {
    interface Interface1,..
    typeof ParentClass,..
    type unsigned,..
 }

You have my vote, for the feature and for the syntax!
Jul 30 2004
parent resistor AT mac DOT com <resistor_member pathlink.com> writes:
 requirements Bob {
    interface Interface1,..
    typeof ParentClass,..
    type unsigned,..
 }

Love it. It covers the complex type filters, and makes the code more readable by abstracting filters. Maybe the keyword should should be filter instead of requirements. -Owen
Jul 30 2004
prev sibling next sibling parent Niko Korhonen <niktheblak hotmail.com> writes:
Walter wrote:

 I hadn't realized that structs and classes cannot be discriminated in
 template specialization, you're right. How does the following look:
     template foo(T : struct)
 and:
     template foo(T : class)
 ?

Oh my god, that's the most beautiful thing I've ever seen! ...in the context of programming languages of course :) It would complement the existing specialization rules so nicely.
Jul 29 2004
prev sibling next sibling parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Walter" <newshound digitalmars.com> wrote in message
news:cebikp$1mh0$1 digitaldaemon.com...
 "Matthew" <admin.hat stlsoft.dot.org> wrote in message
 news:ceaeid$18ir$1 digitaldaemon.com...
 I'm almost at the end of my tether here. There are so many problems with

 seriously whether it's going to be possible to have a decent generic

I think that's to be expected with a new generic language design. After all, the creator of STL (Stepanov?) went through multiple design iterations with Stroustrup. None of us are smart enough to get it all right the first, second, or even third time.

I accept all that. I guess I had to get a bit tabloid to get your attention. ;) Nonetheless, I want us to start out with DTL a level above STL, since we'd be foolish not to learn from our experience with it, no?
You do have a tendency to push the envelope past
 its limits (I see that in your C++ code!), and that's good for finding what
 those limits and weaknesses are.

Hah! So much for me thinking that I use C++ in a sensible, simple, minimalist way. :-(
 Generic programming in D is also going to be in a different style than in
 C++, and use different techniques.

I couldn't agree more.
 We have to carefully think about whether
 the problems are bugs in the compiler,

Definitely
 bugs in the language design,

Probably
 or bugs
 in our thinking

Not a chance. :)
 - perhaps we haven't discovered the right D way to do
 generic programming.

In terms of using and manipulating the elements in containers, as opposed to manipulating containers themselves (i.e. adding/removing/moving elements), then I believe we do have the right approach: foreach. Since I've devised ways to be able to apply filters/transformation to containers that are themselves foreach-able (and can also be subject to filters/transformations themselves), then I believe we are on the right road, or at least one of several right roads. As for manipulating containers themselves, this either needs iterators, or modifying interfaces, to do in a generic manner. The former are impossible/difficult in D, and the latter are, IMO, unattractive. In fact, as a huge devotee of STL, I must say that I virtually never modiy containers generically, beyond using the obvious one of back_inserter. Since DTL containers can be parameterised to derive from interfaces, it's not going to be hard to have them derive from a common interface that supports the add() method. (In fact I played around with this some months, ago, and it worked _reasonably_ well) So, I would (impartially) suggest that if the compiler/language could support what I've been trying to achieve in the last few months, i.e. composable transformations, without the various hassles, then this would be a big step towards container nirvana.
 One thing I am pretty sure of, the C++ way of doing STL
 isn't the right way for D.

I agree.
 I personally find STL to be impenetrably obtuse.

I don't mind STL so much, as I'm reasonably on board with it. And there are a couple of wonderfully powerful yet discoverable STL extension libraries out there ... But I agree that much STL-like code now being produced is totally inacessible. It's little more than intellectual masturbation.
 I refuse to believe that is the best way of doing generics. I think it's
 great that you are taking a different approach than STL does, and are open
 to inventive new ways of thinking about it.

Why thank you. ;)
 Alas, this crashes the compiler. Sigh ...

That's my problem, send me the test case!

That's my problem. There is no test case. There's the DTL implementation. I don't know that I have the skill to trim it down, but I'm absolutely sure I don't have the time. (I'm now doing a rather time-tight contract for a real paying client, so am kind of busy. I spent much of yesterday doing DTL - don't tell my wife!! - so this w/e I'll be playing catch up on my "real" work.) I really think you need to come up with a better way to diagnose flaws in the compiler, such that we can just send you dumps. You're a super smart guy. I know you can do this. And I reckon this'll save masses of time all round in the long run
 4. Iterator based approach. There are two reasons why STL-iterators are

 instantiation, and (ii) there is no operator *(). Neither of these

 little of that - including adaptors for arrays/pointers, but it means

Not having implicit instantiation can make the code a bit more verbose, but it shouldn't be that bad.

You can't write and apply generic code to a given instantion of a container/iterator/range/enumerator without explicit stating its type. That is not generic programming; that's merely code reuse.
 As for operator*(), why should that be a problem?
 Just define a member function called, I'm sure there's a better name,
 operatorStar(), and instead of *foo, type foo.operatorStar().

I musn't have been clear, as several people have made this misunderstanding. The iterator types that are currently coded have the following interface: class SomeIterator { void next() value_type value() int opEquals(class_type rhs) } The point is that without op*(), we cannot support pointers as iterators. As a library implementor, that's actually a blessing, since I can now prescribe that all iterators exhibit certain member types, which means that some of the worst crimes of STL (actually, they're crimes of badly implemented standard libraries, mentioning no names ...) can be avoided.
 1. Beyond a significant, but indefinable level of complexity, the

 symbols, while chopping out any *unrelated* parts makes them locatable

 Nonetheless, that's a bug in the compiler/linker, and doesn't get my blood

 and at this stage a little bugginess is not the crime of the century.

If you can't reduce it, you can't reduce it. Send a reproducible example.
 2. Templates in D are instantiated en masse. What "en masse" actually

 true rules regarding instantation.

The true rule is the whole template is compiled when it is instantiated. (This is unlike C++, which does "lazy instantiation", which means it only compiles the bits of a template that are actually used.)
 But what I can say is that having a template such as the following is

 f**cked:

     template TransformedRange(R, F, T) { class TransformedRange
      : public NotionalRange!(T)
     {
       . . .
         this(R r, F f)
         {
             m_r = r.dup;
             m_f = f;
         }
         this(R r)
         {
             m_r = r.dup;
             m_f = new filter_type();
         }

 Without worrying too much about the details of what a TransformedRange

 not have a default constructor, one cannot instantiate TransformedRange

 *is never called*!

I don't get it from the example given.

The example had a flaw. F === filter_type. (I forgot to change the last filter_type). Basically, if F does not have a default constructor, then the compiler craps on m_f = new F(); << this line
 3. There's no implicit instantiation. This has *massive* consequences,

 - We can't have iterators (as noted above). This is fine for reading from

 "generically" insert into container ranges in the same (or any analogous)

Can you give a canonical example?
 - In order to support the parameterisable interface (e.g.

 way to manipulate built-in types, objects and structs. For some things,

 functions. Alas, there seems to be no way to discriminate structs via

 containers do not support polymorphic interfaces when storing structs.

I hadn't realized that structs and classes cannot be discriminated in template specialization, you're right. How does the following look: template foo(T : struct) and: template foo(T : class) ?

You know me: how it looks is secondary to it working, and not having nasty corner cases. How would such a syntax fit with existing template parameter constraints?
 4. D's import stuff just blows my mind! As a related issue to the boxing

 conflicts between the toString() functions in std.string and my box and

 private imports, to no avail. I concede that this might be my fault, and I

 rules, but as it currently looks to me, it looks bad and stupid.

I suspect the problem you're running into is that there's a toString in std.string, and a toString in foo. Then, in bar.d: import std.string; import foo; ... x = toString(y); ... This will bring up a conflict, because which toString do you want, the one in std.string or the one in foo? The compiler doesn't know. The rule is that name lookup happens first, *then* overload matching. If you want to overload them together: import std.string; import foo; alias std.string.toString toString; alias foo.toString toString; ... x = toString(y); ... now there's no longer a conflict, because toString is found in the current module (the aliases), and the aliases are chased down for overload resolution. It is this way because it gives you complete control over overloading.

Hmm. I'll have to think about that one after my morning ride. My first reaction is a what? with a capital W
 So, what do we do about this? Walter's stock response is to boil it down,

 non-trivial projects such as DTL, or I'm simply not smart enough to do it.

 of a too-complex design, but I think that's crap: the concept is simple,

 (Very similar techniques that I've experimented on in C++ work fine.)

If you can't do it, you can't do it. Send what you can do.
 Rather than having things boiled down, I think the compiler should be

 information, so we can email a dump of that to Walter, and which will be

Trust me, sending me dumps would be quite useless.

Jul 29 2004
next sibling parent James Widman <james jwidman.com> writes:
In article <cebr2j$1qic$1 digitaldaemon.com>,
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote:

 "Walter" <newshound digitalmars.com> wrote in message 
 news:cebikp$1mh0$1 digitaldaemon.com...
 "Matthew" <admin.hat stlsoft.dot.org> wrote in message
 news:ceaeid$18ir$1 digitaldaemon.com...
 Alas, this crashes the compiler. Sigh ...

That's my problem, send me the test case!

That's my problem. There is no test case. There's the DTL implementation. I don't know that I have the skill to trim it down, but I'm absolutely sure I don't have the time.

I think Walter was implying (maybe not on that line of text, but elsewhere) that if you have anything that elicits bad behavior in the compiler -- regardless of how much trimming it may need -- then you have a test case.
 I really think you need to come up with a better way to diagnose flaws in the 
 compiler, such that we can just send you
 dumps. You're a super smart guy. I know you can do this. And I reckon this'll 
 save masses of time all round in the long
 run

Well, ok, but how practical is it for walter to sift through voluminous debugging output to try to find a bug whose general case (i.e., the boiled-down version that elicits the buggy behavior) is some phantom about which we have no substantial clue? Without an example (*any* example, even if it can't be boiled down), it's difficult (if not impossible) to know what to look for. What kind of turnaround time could we then expect for bugfixes? Come to think of it, we probably wouldn't speak of a "turnaround time for bugfixes," because there would only be one bug, and it would consume all of Walter's time. It would become known simply as "The Bug." Hopefully we would see it, and the next DMD point release, sometime before the ISO C++ committee releases the next Standard. :-) Seriously though, how can anyone reasonably expect to solve a problem this complex without stepping into the compiler's code with a debugger? Suppose Walter finds a bug without a D source sample; how can he know whether it was the right one? And how can he know if he's solved the general case? All of these reasons, and possibly more, are why Walter gave this succinct response:
 Trust me, sending me dumps would be quite useless.


Jul 29 2004
prev sibling next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
news:cebr2j$1qic$1 digitaldaemon.com...
 or bugs
 in our thinking


Bugs in our thinking are very likely.
 - perhaps we haven't discovered the right D way to do
 generic programming.

In terms of using and manipulating the elements in containers, as opposed

 adding/removing/moving elements), then I believe we do have the right

 Since I've devised ways to be able to apply filters/transformation to

 can also be subject to filters/transformations themselves), then I believe

 several right roads.

I think we might be, too. More experience will tell, so far there have been no users of your design. I expect that experience will force changes, just as experience has forced changes in D and some rethinking of some of my cherished erroneous ideas. Some of the Wright brothers' ideas have survived into modern aircraft, many were scrapped as obviously (in hindsight) the wrong way. That funky hip control for roll was one of those <g>.
 Alas, this crashes the compiler. Sigh ...



Of course you have a test case. The whole thing, if need be. Hopefully, at least provide a .bat file that will result in the crash.
 I really think you need to come up with a better way to diagnose flaws in

 dumps.

I've been at this game for decades. There is no better way than having the code that tickles the tiger.
 The iterator types that are currently coded have the following interface:

     class SomeIterator
     {
             void next()
             value_type value()
             int opEquals(class_type rhs)
     }

 The point is that without op*(), we cannot support pointers as iterators.

 blessing, since I can now prescribe that all iterators exhibit certain

 crimes of STL (actually, they're crimes of badly implemented standard

 avoided.

D tries to move away from using pointers, so I think not directly supporting pointers as iterators might be a good idea. But remember, a pointer can be wrapped in a struct: struct Pointer(T) { T* ptr; void next() value_type value() int opEquals(class_type rhs) } and used that way.
 Basically, if F does not have a default constructor, then the compiler

              m_f = new F(); << this line

I think I can fix that.
Jul 29 2004
parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Walter" <newshound digitalmars.com> wrote in message
news:cecrat$2830$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in message
 news:cebr2j$1qic$1 digitaldaemon.com...
 or bugs
 in our thinking


Bugs in our thinking are very likely.
 - perhaps we haven't discovered the right D way to do
 generic programming.

In terms of using and manipulating the elements in containers, as opposed

 adding/removing/moving elements), then I believe we do have the right

 Since I've devised ways to be able to apply filters/transformation to

 can also be subject to filters/transformations themselves), then I believe

 several right roads.

I think we might be, too. More experience will tell, so far there have been no users of your design.

Well, that's because I've not really been able to get it stably compilable. The bits that do work at the moment are hardly breathtaking in vision.
 I expect that experience will force changes, just
 as experience has forced changes in D and some rethinking of some of my
 cherished erroneous ideas. Some of the Wright brothers' ideas have survived
 into modern aircraft, many were scrapped as obviously (in hindsight) the
 wrong way. That funky hip control for roll was one of those <g>.


 Alas, this crashes the compiler. Sigh ...



Of course you have a test case. The whole thing, if need be. Hopefully, at least provide a .bat file that will result in the crash.

I will do so very before the end of the w/e.
 I really think you need to come up with a better way to diagnose flaws in

 dumps.

I've been at this game for decades. There is no better way than having the code that tickles the tiger.

I agree. My objection was your requirement for boiled down cases. Sometimes it's just not practical, and sometimes it's just not possible. (I concede that the remainder of circumstances boil down to laziness.)
 The iterator types that are currently coded have the following interface:

     class SomeIterator
     {
             void next()
             value_type value()
             int opEquals(class_type rhs)
     }

 The point is that without op*(), we cannot support pointers as iterators.

 blessing, since I can now prescribe that all iterators exhibit certain

 crimes of STL (actually, they're crimes of badly implemented standard

 avoided.

D tries to move away from using pointers, so I think not directly supporting pointers as iterators might be a good idea.

Same here.
 But remember, a pointer can be
 wrapped in a struct:

 struct Pointer(T)
 {        T* ptr;
             void next()
             value_type value()
             int opEquals(class_type rhs)
  }

 and used that way.

Of course. That's all planned. :)
 Basically, if F does not have a default constructor, then the compiler

              m_f = new F(); << this line

I think I can fix that.

If you can, I will kiss you. :) I shall try and start funnelling stuff your way shortly.
Jul 30 2004
parent "Walter" <newshound digitalmars.com> writes:
"Matthew" <admin.hat stlsoft.dot.org> wrote in message
news:cecug0$2ag9$1 digitaldaemon.com...
 I think I can fix that.

If you can, I will kiss you. :)

That would get me in trouble. I recommend instead sending large quantities of cash.
Jul 30 2004
prev sibling parent Farmer <itsFarmer. freenet.de> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> wrote in
news:cebr2j$1qic$1 digitaldaemon.com: 

 But what I can say is that having a template such as the following is

 f**cked:

     template TransformedRange(R, F, T) { class TransformedRange
      : public NotionalRange!(T)
     {
       . . .
         this(R r, F f)
         {
             m_r = r.dup;
             m_f = f;
         }
         this(R r)
         {
             m_r = r.dup;



             m_f = new F();  
         }
     }



It seems to me that this 'all or nothing' approach of D templates is a bit too restrictive. So here's my suggestion to rectify this. I suggest the attribute 'optional'. The optional attribute can be applied to any member (variables, functions, aliases, classes, members within classes etc.) of a template body. It means that a template instance *may* but *need not* 'support' the template member. Effectively, a template declaration consists of a set of guaranteed members and optional members. <example> template TFoo(T) { uint length(T v) { return v.length; } // Removes an element from an array. // Note: Function is only allowed for dynamic arrays and AAs. optional void remove(inout T array, int index) { delete array[index]; } optional alias T.bar SomeAlias; } ... int[4] fixed; int[] dynamic=new int[4]; TFoo!(int[4]).length(fixed); TFoo!(int[]).length(dynamic); TFoo!(int[]).remove(dynamic, 1); // works TFoo!(int[4]).remove(fixed); // Error: optional function remove(inout int[4], int) not supported by instance TFoo!(int[4]). Reason: cannot have out or inout parameter of type int[4] TFoo!(int).length(5); // Error: no property 'length' for type 'int' </example> Explanation: When the template is instantiated the compiler looks for semantic errors of all template members. The instantiation fails, if a semantic error is caused by a garanteed template member (AFAIK, this is the current way that D templates work). Any semantic errors that are caused by an optional template member, merely means that the specific template instance does not support the 'erroneous' optional template member. It is an error if a garanteed template member depends on an optional template member, even if the specific template instance does support the optional template member. Therefore an optional member of template declaration can only be accessed if a) the specific template instance supports the optional member. or b) the optional template member is accessed within the context of an optional template member of a template body. Why is lazy template-evalution sometimes acceptable? The problem in the given (contrived) example could be easily solved using template specialization. But for big generic libraries, tons of specializations, interfaces and template meta programming tricks might be necessary. This is a burden for designers of the generic libraries and its users. When writting proof-of-concept code, the designers are pestered by an endless stream of errors from the pedantic compiler. And when the library is finally ready (late, of course) its users have to figure out the peculiarities of a dozens of specializations, which results in a steep-learning curve. Preferably, the use of optional elements in templates would be avoided by library designers, but for some cases the added simplicity of the library is well worth the costs of an occasional obscure error message for the user. Disclaimer: I made everything out of thin air. Therefore none of my statements could refer to the DTL or any simliar work. Any, thoughts? Farmer.
Aug 02 2004
prev sibling parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Walter wrote:

 
 "Matthew" <admin.hat stlsoft.dot.org> wrote in message
 news:ceaeid$18ir$1 digitaldaemon.com...
 I'm almost at the end of my tether here. There are so many problems with

 seriously whether it's going to be possible to have a decent generic

I think that's to be expected with a new generic language design. After all, the creator of STL (Stepanov?) went through multiple design iterations with Stroustrup. None of us are smart enough to get it all right the first, second, or even third time. You do have a tendency to push the envelope past its limits (I see that in your C++ code!), and that's good for finding what those limits and weaknesses are. Generic programming in D is also going to be in a different style than in C++, and use different techniques. We have to carefully think about whether the problems are bugs in the compiler, bugs in the language design, or bugs in our thinking - perhaps we haven't discovered the right D way to do generic programming.

Now, this sounds like D has become an experimental language after all. If there is no grand concept yet of how generic programming should be done, we are just going down the same road that C++ went: templates were introduced and lateron, when people had found ways to really exploit them, they found their limitations, but it was too late to change. Now, for D, if it is left to the future to re-discover generic programming from ground up, we are very likely to find out that the current concept of templates is sub-optimal as well. Are we really ready for 1.0 before we know where we want to go in that important area? I think it is rather dangerous to separate the development of the language and the standard library so much. The language should certainly not be called 1.0 before we have a clear concept of how the DTL should be designed. A 1.0 language without an 1.0 library isn't worth much anyway.
Jul 29 2004
parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:cecmf7$26e2$1 digitaldaemon.com...
 Walter wrote:

 "Matthew" <admin.hat stlsoft.dot.org> wrote in message
 news:ceaeid$18ir$1 digitaldaemon.com...
 I'm almost at the end of my tether here. There are so many problems with

 seriously whether it's going to be possible to have a decent generic

I think that's to be expected with a new generic language design. After all, the creator of STL (Stepanov?) went through multiple design iterations with Stroustrup. None of us are smart enough to get it all right the first, second, or even third time. You do have a tendency to push the envelope past its limits (I see that in your C++ code!), and that's good for finding what those limits and weaknesses are. Generic programming in D is also going to be in a different style than in C++, and use different techniques. We have to carefully think about whether the problems are bugs in the compiler, bugs in the language design, or bugs in our thinking - perhaps we haven't discovered the right D way to do generic programming.

Now, this sounds like D has become an experimental language after all. If there is no grand concept yet of how generic programming should be done, we are just going down the same road that C++ went: templates were introduced and lateron, when people had found ways to really exploit them, they found their limitations, but it was too late to change.

Aren't we going down the rod in the opposite direction? We know what we want, but we don't have the (template) tools to get it?
 Now, for D, if it is left to the future to re-discover generic programming
 from ground up, we are very likely to find out that the current concept of
 templates is sub-optimal as well. Are we really ready for 1.0 before we
 know where we want to go in that important area?

 I think it is rather dangerous to separate the development of the language
 and the standard library so much. The language should certainly not be
 called 1.0 before we have a clear concept of how the DTL should be
 designed. A 1.0 language without an 1.0 library isn't worth much anyway.

I agree with these two paras.
Jul 30 2004
parent Norbert Nemec <Norbert Nemec-online.de> writes:
Matthew wrote:

 
 "Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
 Now, this sounds like D has become an experimental language after all. If
 there is no grand concept yet of how generic programming should be done,
 we are just going down the same road that C++ went: templates were
 introduced and lateron, when people had found ways to really exploit
 them, they found their limitations, but it was too late to change.

Aren't we going down the rod in the opposite direction? We know what we want, but we don't have the (template) tools to get it?

Looking at Walters messages it does not look like this. In many cases when he rejects feature requests concerning generic programming, it boils down to "you are thinking C++, start thinking D instead" - unfortunately, looking at the message I was replying to, it seems not at all clear what the D approach to generic programming is.
Jul 30 2004
prev sibling next sibling parent reply Charlie <Charlie_member pathlink.com> writes:
I personally think generics are over-rated, if i wanted typeless variables i'd
use a scripting language ( or VB ).


    template Vector(T, B = EmptyBase)
    {
        class Vector
            : public BaseSelector!(B).selected_type

I realize you do alot of C++ and D programming, so you want to keep the syntax roughly the same, but this looks dangerously close to C++. Why noy use the D way :) class Vector(T,B = EmptyBase) : BaseSelector!(B).selected_type I only mention this because I wonder how much ( unconsiously ) your applying C++'s soloutions to D. I for one am investing a large amount of
time to the D cause, with all the consequences wrt to more materially lucrative
activities. Since I find that boiling
down the code generally takes me at least as long to write the code in the
first place, I just don't see that it's
justified that I should do the boiling as well. Give me a -dump switch, and I
can send it all off to Walter.

I hear you on that one.
[btw: both of the above forms work now and, fwiw, I hope to release some of the
containers supporting this in the next
c/o days.]

Youve been saying that for months ;). But im not downing you I know your busy and all good things take time, but _please_ do release it , so that we can look at it too, theres lots of talent on the newsgroup.
4. D's import stuff just blows my mind!

Maybe we can convince walter to give us a written and posted on the website description, as this seems to be a problem everyone has had at one time or another. I know hes answered it a few times, but it needs to be easily accessible.
If all this sounds like I'm down on D, and I've spent several weeks - actually
it's several months, but who's
counting? - working against all these issues and more, then you'd be about spot
on. I can't remember being this
technically frustrated in the last 10 years, and I'm generally known as a
practical pragmatist! :-(

Well i hope you don't give up, hate to lose you at this stage, but I feel your frustration :S. Charlie In article <ceaeid$18ir$1 digitaldaemon.com>, Matthew says...
I'm almost at the end of my tether here. There are so many problems with the
language/compiler, that I'm wondering
seriously whether it's going to be possible to have a decent generic template
library.

Before I preface this, I should reiterate the "modes" of DTL, to inform on any
thoughts anyone might have on this:

1. foreach - client code uses foreach. This all works well for all containers.
The problem with this is only that it's
an unsophisticated view of looking at a container: you get everything, and have
to do all filtering and transformations
yourself in client code. This can result in verbose and tedious coding. Good
libraries should provide more power.

2. Transformations/filtering - based on "Ranges" (a concept John Torjo and I
concocted for C++, but which is eminently
suitable for D.). Basically, each container provides a set of methods
(eventually via a mixin) that returns a "range".
For example, the "select()" method takes either a function/delegate/functor
predicate, and returns a freachable range
that "contains" only the elements matching the predicate, e.g.

    List!(int)        list = . . .; // Assume it's filled with numbers 0 - 9
    bool IsOdd(int i)
    {
        return 0 != (i & 0x2);
    }
    foreach(int i; list.select(IsOdd))
    {
        printf("%d ", i);
    }

This will print out "1 3 5 7 9"

The power of this is that the returned "range" itself provides the same
transformation methods, which means that the
transformations are composable, as in:

    List!(int)        list = . . .; // Assume it's filled with numbers 0 - 9
    bool IsOdd(int i)
    {
        return 0 != (i & 0x2);
    }
    int Times2(int i)
    {
        return 2 * i;
    }
    foreach(int i; list.select(IsOdd).collect(Times2))
    {
        printf("%d ", i);
    }

This will print out "2 6 10 14 18"

I'm sure you can see the power in this approach, as well as the potential
efficiency savings, where the transformation
"criteria" may be tunnelled into the original container's foreach (I've not
done that yet).

3. Interface-based containers

Each container in DTL is declared as follows:

    template Vector(T, B = EmptyBase)
    {
        class Vector
            : public BaseSelector!(B).selected_type


This means that by default the container will be do as the STL does, and will
just be all compile-time typing, e.g.

    alias    Vector!(int)     int_vector_t;

    int_vector_t    v = new ...;

    v[10] = 3;
    size_t n = v.length;

    int    r = v[0] + v[1];

Such a type is great, as long as we want to manipulate it in/with code that
knows its exact type.

Alternatively, we can also support the old-style Java approach, whereby one can
specify a base interface, e.g.

    void dump(IObjectContainer c)
    {
        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
        {
            printf("%.*s ", e.nextElement().toString());
        }
        printf("\n");
    }

    alias    Vector!(int, IObjectContainer)    int_vector_IObjectContainer_t;
    alias    List!(int, IObjectContainer)        int_list_IObjectContainer_t;

    int_vector_IObjectContainer_t   v = new ...;
    int_list_IObjectContainer_t        l = new ...;

    dump(v);
    dump(l);

Now we can pass instances of any container implementing IObjectContainer this
to *any* function that knows how to
manipulate that interface.

[btw: both of the above forms work now and, fwiw, I hope to release some of the
containers supporting this in the next
c/o days.]

The downside to the IObjectContainer type approach is that (i) fundamental
types must be "boxed" (I've already worked up
std.box), and (ii) it's not generic, since one must downcast object types to
their "real" types from Object. Pretty
pukey stuff, except in the minority of circumstances where the IObject-stuff is
what you're after (e.g. a message
board).

Further to these two approaches, which I've been working on today, is
inheritance via parameterised-interfaces, e.g.

    template dump(T) { void dump(IContainer!(T) c)
    {
        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
        {
            T    t    =    e.nextElement();

            // ... write generically, presumably via writef() (which I've not
used yet <g>)
        }
        printf("\n");
    }

    alias    IContainer!(int)                         int_IContainer_t;
    alias    Vector!(int, int_IContainer_t)    int_vector_IContainer_int_t;
    alias    List!(int, int_IContainer_t)        int_list_IContainer_int_t;

    int_vector_IContainer_int_t      v = new ...;
    int_list_IContainer_int_t            l = new ...;

    dump(v);
    dump(l);

Note: this approach does _not_ require boxing or downcasting.

Alas, this crashes the compiler. Sigh ...

4. Iterator based approach. There are two reasons why STL-iterators are not
possible in D: (i) there is no implicit
instantiation, and (ii) there is no operator *(). Neither of these preclude the
creation of iterators - and I've done a
little of that - including adaptors for arrays/pointers, but it means they're
not really very usable. The only way to
use them "generically" in algorithms would be polymorphically if the iterator
classes derived from interfaces, and that
would preclude efficiency mechanisms (such as Random Access iterator
advancement in STL), without some nasty hacks.

It's my belief that iterators are not for D, and that we thus have the modes
1-3 described above.

And now on to my litany of woes ...

Some tasters:

1. Beyond a significant, but indefinable level of complexity, the
linker/compiler loose the ability to find all required
symbols, while chopping out any *unrelated* parts makes them locatable again.
Boil it down? I wish!!

Nonetheless, that's a bug in the compiler/linker, and doesn't get my blood up,
since one can usually find workarounds,
and at this stage a little bugginess is not the crime of the century.

2. Templates in D are instantiated en masse. What "en masse" actually means is
beyond me, since I am yet to work out the
true rules regarding instantation. But what I can say is that having a template
such as the following is totally
f**cked:

    template TransformedRange(R, F, T) { class TransformedRange
     : public NotionalRange!(T)
    {
      . . .
        this(R r, F f)
        {
            m_r = r.dup;
            m_f = f;
        }
        this(R r)
        {
            m_r = r.dup;
            m_f = new filter_type();
        }

Without worrying too much about the details of what a TransformedRange does,
the problem is pretty obvious. If F does
not have a default constructor, one cannot instantiate TransformedRange even in
the case where the single parameter ctor
*is never called*!

I'm yet to work out a workaround to this one - all I do at the moment is not
use certain operations with certain
container instantiations in the test programs. Woo hoo!

3. There's no implicit instantiation. This has *massive* consequences,
including:

- We can't have iterators (as noted above). This is fine for reading from
containers, but consider how we might
"generically" insert into container ranges in the same (or any analogous) way
as is the case with STL.

- In order to support the parameterisable interface (e.g. IContainer!(int))
described above, there needs to be a common
way to manipulate built-in types, objects and structs. For some things, one can
use traits, for others, overloaded
functions. Alas, there seems to be no way to discriminate structs via
overloaded functions. Hence, currently the DTL
containers do not support polymorphic interfaces when storing structs.
Naturally, this has much wider consequences

4. D's import stuff just blows my mind! As a related issue to the boxing
utility class described above, I'm running into
conflicts between the toString() functions in std.string and my box and boxutil
modules. I've tried all kinds of use of
private imports, to no avail. I concede that this might be my fault, and I
might have just failed to grok D's import
rules, but as it currently looks to me, it looks bad and stupid.


So, what do we do about this? Walter's stock response is to boil it down, but
it's either impossible to do so with
non-trivial projects such as DTL, or I'm simply not smart enough to do it.
Sure, one might argue that this is indicative
of a too-complex design, but I think that's crap: the concept is simple, and
the code is simple; it just doesn't work.
(Very similar techniques that I've experimented on in C++ work fine.)

Rather than having things boiled down, I think the compiler should be amended
to provide *copious" debugging
information, so we can email a dump of that to Walter, and which will be useful
to him. I don't know what that
information should be, but I know that it's simply not practical for me, or
anyone else, to "boil down" these
precipitating bugs when they only manifest in highly complex code. I do know
that it's totally bogus for us to be made
to feel guilty for not having the time or the talent to do this boiling down. I
for one am investing a large amount of
time to the D cause, with all the consequences wrt to more materially lucrative
activities. Since I find that boiling
down the code generally takes me at least as long to write the code in the
first place, I just don't see that it's
justified that I should do the boiling as well. Give me a -dump switch, and I
can send it all off to Walter.

Anyway, that's just the compiler, and my main problem is with the language. I'm
coming to the conclusion that D either
will never be suitable for generic programming, or such suitability is years
away. Given that, my aims for DTL are
starting to seem naive at best, unattainable at worst.

What's to be done? Well, one might say let's just have vanilla containers, and
skip all the transformation stuff. That's
fine, but then where're the generics? We can't have algorithms, remember,
because we've not got implicit instantiation!
The only remaining way to be generic is to follow the Java-route, and go with
polymorphic container interfaces, but (i)
they can't contain structures, and (ii) we're in Java-la-la land where
everything has to be downcast. Yeuch! Even if we
can get the compiler to accept parameterisable container interfaces, it's still
a runtme indirection, with the
concomitant efficiency costs.

So please, someone enlighten me (since I am quite prepared to believe I've
missed something simple and obvious here):
how can we do generic programming in D?

Matthew

If all this sounds like I'm down on D, and I've spent several weeks - actually
it's several months, but who's
counting? - working against all these issues and more, then you'd be about spot
on. I can't remember being this
technically frustrated in the last 10 years, and I'm generally known as a
practical pragmatist! :-(

Jul 29 2004
parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Charlie" <Charlie_member pathlink.com> wrote in message
news:cebotu$1plh$1 digitaldaemon.com...
 I personally think generics are over-rated, if i wanted typeless variables i'd
 use a scripting language ( or VB ).

Is this related to something in my post (if so what?), or just a statement of your position?
    template Vector(T, B = EmptyBase)
    {
        class Vector
            : public BaseSelector!(B).selected_type

I realize you do alot of C++ and D programming, so you want to keep the syntax roughly the same, but this looks dangerously close to C++. Why noy use the D way :) class Vector(T,B = EmptyBase) : BaseSelector!(B).selected_type

I do a lot of programming in a lot of languages, and I always lay out my inheritance lists in the same way. AFAIAC it's got nothing specifically to do with C++.
 I only mention this because I wonder how much ( unconsiously ) your applying
 C++'s soloutions to D.

Undoubtedly. I am also _consciously_ applying ideas from many languages. Is that a problem? Since D doesn't yet have established methodologies for all this kind of thing - apart from foreach - what would you suggest instead?
 I for one am investing a large amount of
time to the D cause, with all the consequences wrt to more materially lucrative
activities. Since I find that boiling
down the code generally takes me at least as long to write the code in the
first place, I just don't see that it's
justified that I should do the boiling as well. Give me a -dump switch, and I
can send it all off to Walter.

I hear you on that one.
[btw: both of the above forms work now and, fwiw, I hope to release some of the
containers supporting this in the


c/o days.]

Youve been saying that for months ;). But im not downing you I know your busy and all good things take time, but _please_ do release it , so that we can look at it too, theres lots of talent on the newsgroup.

This w/e, I promise.
4. D's import stuff just blows my mind!

Maybe we can convince walter to give us a written and posted on the website description, as this seems to be a problem everyone has had at one time or another. I know hes answered it a few times, but it needs to be easily accessible.

Agreed
If all this sounds like I'm down on D, and I've spent several weeks - actually
it's several months, but who's
counting? - working against all these issues and more, then you'd be about spot
on. I can't remember being this
technically frustrated in the last 10 years, and I'm generally known as a
practical pragmatist! :-(

Well i hope you don't give up, hate to lose you at this stage, but I feel your frustration :S.

Don't really know how to give up. Probably a character flaw, that ..
 Charlie


 In article <ceaeid$18ir$1 digitaldaemon.com>, Matthew says...
I'm almost at the end of my tether here. There are so many problems with the
language/compiler, that I'm wondering
seriously whether it's going to be possible to have a decent generic template
library.

Before I preface this, I should reiterate the "modes" of DTL, to inform on any
thoughts anyone might have on this:

1. foreach - client code uses foreach. This all works well for all containers.
The problem with this is only that


an unsophisticated view of looking at a container: you get everything, and have
to do all filtering and


yourself in client code. This can result in verbose and tedious coding. Good
libraries should provide more power.

2. Transformations/filtering - based on "Ranges" (a concept John Torjo and I
concocted for C++, but which is


suitable for D.). Basically, each container provides a set of methods
(eventually via a mixin) that returns a


For example, the "select()" method takes either a function/delegate/functor
predicate, and returns a freachable range
that "contains" only the elements matching the predicate, e.g.

    List!(int)        list = . . .; // Assume it's filled with numbers 0 - 9
    bool IsOdd(int i)
    {
        return 0 != (i & 0x2);
    }
    foreach(int i; list.select(IsOdd))
    {
        printf("%d ", i);
    }

This will print out "1 3 5 7 9"

The power of this is that the returned "range" itself provides the same
transformation methods, which means that the
transformations are composable, as in:

    List!(int)        list = . . .; // Assume it's filled with numbers 0 - 9
    bool IsOdd(int i)
    {
        return 0 != (i & 0x2);
    }
    int Times2(int i)
    {
        return 2 * i;
    }
    foreach(int i; list.select(IsOdd).collect(Times2))
    {
        printf("%d ", i);
    }

This will print out "2 6 10 14 18"

I'm sure you can see the power in this approach, as well as the potential
efficiency savings, where the


"criteria" may be tunnelled into the original container's foreach (I've not
done that yet).

3. Interface-based containers

Each container in DTL is declared as follows:

    template Vector(T, B = EmptyBase)
    {
        class Vector
            : public BaseSelector!(B).selected_type


This means that by default the container will be do as the STL does, and will
just be all compile-time typing, e.g.

    alias    Vector!(int)     int_vector_t;

    int_vector_t    v = new ...;

    v[10] = 3;
    size_t n = v.length;

    int    r = v[0] + v[1];

Such a type is great, as long as we want to manipulate it in/with code that
knows its exact type.

Alternatively, we can also support the old-style Java approach, whereby one can
specify a base interface, e.g.

    void dump(IObjectContainer c)
    {
        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
        {
            printf("%.*s ", e.nextElement().toString());
        }
        printf("\n");
    }

    alias    Vector!(int, IObjectContainer)    int_vector_IObjectContainer_t;
    alias    List!(int, IObjectContainer)        int_list_IObjectContainer_t;

    int_vector_IObjectContainer_t   v = new ...;
    int_list_IObjectContainer_t        l = new ...;

    dump(v);
    dump(l);

Now we can pass instances of any container implementing IObjectContainer this
to *any* function that knows how to
manipulate that interface.

[btw: both of the above forms work now and, fwiw, I hope to release some of the
containers supporting this in the


c/o days.]

The downside to the IObjectContainer type approach is that (i) fundamental
types must be "boxed" (I've already worked


std.box), and (ii) it's not generic, since one must downcast object types to
their "real" types from Object. Pretty
pukey stuff, except in the minority of circumstances where the IObject-stuff is
what you're after (e.g. a message
board).

Further to these two approaches, which I've been working on today, is
inheritance via parameterised-interfaces, e.g.

    template dump(T) { void dump(IContainer!(T) c)
    {
        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
        {
            T    t    =    e.nextElement();

            // ... write generically, presumably via writef() (which I've not
used yet <g>)
        }
        printf("\n");
    }

    alias    IContainer!(int)                         int_IContainer_t;
    alias    Vector!(int, int_IContainer_t)    int_vector_IContainer_int_t;
    alias    List!(int, int_IContainer_t)        int_list_IContainer_int_t;

    int_vector_IContainer_int_t      v = new ...;
    int_list_IContainer_int_t            l = new ...;

    dump(v);
    dump(l);

Note: this approach does _not_ require boxing or downcasting.

Alas, this crashes the compiler. Sigh ...

4. Iterator based approach. There are two reasons why STL-iterators are not
possible in D: (i) there is no implicit
instantiation, and (ii) there is no operator *(). Neither of these preclude the
creation of iterators - and I've done


little of that - including adaptors for arrays/pointers, but it means they're
not really very usable. The only way to
use them "generically" in algorithms would be polymorphically if the iterator
classes derived from interfaces, and


would preclude efficiency mechanisms (such as Random Access iterator
advancement in STL), without some nasty hacks.

It's my belief that iterators are not for D, and that we thus have the modes
1-3 described above.

And now on to my litany of woes ...

Some tasters:

1. Beyond a significant, but indefinable level of complexity, the
linker/compiler loose the ability to find all


symbols, while chopping out any *unrelated* parts makes them locatable again.
Boil it down? I wish!!

Nonetheless, that's a bug in the compiler/linker, and doesn't get my blood up,
since one can usually find


and at this stage a little bugginess is not the crime of the century.

2. Templates in D are instantiated en masse. What "en masse" actually means is
beyond me, since I am yet to work out


true rules regarding instantation. But what I can say is that having a template
such as the following is totally
f**cked:

    template TransformedRange(R, F, T) { class TransformedRange
     : public NotionalRange!(T)
    {
      . . .
        this(R r, F f)
        {
            m_r = r.dup;
            m_f = f;
        }
        this(R r)
        {
            m_r = r.dup;
            m_f = new filter_type();
        }

Without worrying too much about the details of what a TransformedRange does,
the problem is pretty obvious. If F does
not have a default constructor, one cannot instantiate TransformedRange even in
the case where the single parameter


*is never called*!

I'm yet to work out a workaround to this one - all I do at the moment is not
use certain operations with certain
container instantiations in the test programs. Woo hoo!

3. There's no implicit instantiation. This has *massive* consequences,
including:

- We can't have iterators (as noted above). This is fine for reading from
containers, but consider how we might
"generically" insert into container ranges in the same (or any analogous) way
as is the case with STL.

- In order to support the parameterisable interface (e.g. IContainer!(int))
described above, there needs to be a


way to manipulate built-in types, objects and structs. For some things, one can
use traits, for others, overloaded
functions. Alas, there seems to be no way to discriminate structs via
overloaded functions. Hence, currently the DTL
containers do not support polymorphic interfaces when storing structs.
Naturally, this has much wider consequences

4. D's import stuff just blows my mind! As a related issue to the boxing
utility class described above, I'm running


conflicts between the toString() functions in std.string and my box and boxutil
modules. I've tried all kinds of use


private imports, to no avail. I concede that this might be my fault, and I
might have just failed to grok D's import
rules, but as it currently looks to me, it looks bad and stupid.


So, what do we do about this? Walter's stock response is to boil it down, but
it's either impossible to do so with
non-trivial projects such as DTL, or I'm simply not smart enough to do it.
Sure, one might argue that this is


of a too-complex design, but I think that's crap: the concept is simple, and
the code is simple; it just doesn't


(Very similar techniques that I've experimented on in C++ work fine.)

Rather than having things boiled down, I think the compiler should be amended
to provide *copious" debugging
information, so we can email a dump of that to Walter, and which will be useful
to him. I don't know what that
information should be, but I know that it's simply not practical for me, or
anyone else, to "boil down" these
precipitating bugs when they only manifest in highly complex code. I do know
that it's totally bogus for us to be


to feel guilty for not having the time or the talent to do this boiling down. I
for one am investing a large amount


time to the D cause, with all the consequences wrt to more materially lucrative
activities. Since I find that boiling
down the code generally takes me at least as long to write the code in the
first place, I just don't see that it's
justified that I should do the boiling as well. Give me a -dump switch, and I
can send it all off to Walter.

Anyway, that's just the compiler, and my main problem is with the language. I'm
coming to the conclusion that D


will never be suitable for generic programming, or such suitability is years
away. Given that, my aims for DTL are
starting to seem naive at best, unattainable at worst.

What's to be done? Well, one might say let's just have vanilla containers, and
skip all the transformation stuff.


fine, but then where're the generics? We can't have algorithms, remember,
because we've not got implicit


The only remaining way to be generic is to follow the Java-route, and go with
polymorphic container interfaces, but


they can't contain structures, and (ii) we're in Java-la-la land where
everything has to be downcast. Yeuch! Even if


can get the compiler to accept parameterisable container interfaces, it's still
a runtme indirection, with the
concomitant efficiency costs.

So please, someone enlighten me (since I am quite prepared to believe I've
missed something simple and obvious here):
how can we do generic programming in D?

Matthew

If all this sounds like I'm down on D, and I've spent several weeks - actually
it's several months, but who's
counting? - working against all these issues and more, then you'd be about spot
on. I can't remember being this
technically frustrated in the last 10 years, and I'm generally known as a
practical pragmatist! :-(


Jul 30 2004
parent reply "Carlos Santander B." <carlos8294 msn.com> writes:
"Matthew" <admin.hat stlsoft.dot.org> escribió en el mensaje
news:cecug1$2ag9$2 digitaldaemon.com
| "Charlie" <Charlie_member pathlink.com> wrote in message
| news:cebotu$1plh$1 digitaldaemon.com...
|||    template Vector(T, B = EmptyBase)
|||    {
|||        class Vector
|||            : public BaseSelector!(B).selected_type
||
|| I realize you do alot of C++ and D programming, so you want to keep the
syntax
|| roughly the same, but this looks dangerously close to C++.  Why noy use the D
|| way :)
||
|| class Vector(T,B = EmptyBase) : BaseSelector!(B).selected_type
|
| I do a lot of programming in a lot of languages, and I always lay out my
| inheritance lists in the same way. AFAIAC it's got nothing specifically to do
with
| C++.
|

What Charlie meant there was that you don't need the "template" keyword if
you're only defining a templated class. Instead you could use the short syntax:
class C(T) {...} Also, "public" is default in D. I don't know if that holds true
for inheritance, though. If it's also by default there, then it wouldn't be
neede in your example.

That relates to what he said later:

|| I only mention this because I wonder how much ( unconsiously ) your applying
|| C++'s soloutions to D.
|
| Undoubtedly. I am also _consciously_ applying ideas from many languages. Is
that a
| problem? Since D doesn't yet have established methodologies for all this kind
of
| thing - apart from foreach - what would you suggest instead?

-----------------------
Carlos Santander Bernal
Jul 30 2004
parent "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Carlos Santander B." <carlos8294 msn.com> wrote in message
news:ceeigl$74j$1 digitaldaemon.com...
 "Matthew" <admin.hat stlsoft.dot.org> escribió en el mensaje
 news:cecug1$2ag9$2 digitaldaemon.com
 | "Charlie" <Charlie_member pathlink.com> wrote in message
 | news:cebotu$1plh$1 digitaldaemon.com...
 |||    template Vector(T, B = EmptyBase)
 |||    {
 |||        class Vector
 |||            : public BaseSelector!(B).selected_type
 ||
 || I realize you do alot of C++ and D programming, so you want to keep the
 syntax
 || roughly the same, but this looks dangerously close to C++.  Why noy use the
D
 || way :)
 ||
 || class Vector(T,B = EmptyBase) : BaseSelector!(B).selected_type
 |
 | I do a lot of programming in a lot of languages, and I always lay out my
 | inheritance lists in the same way. AFAIAC it's got nothing specifically to do
 with
 | C++.
 |

 What Charlie meant there was that you don't need the "template" keyword if
 you're only defining a templated class. Instead you could use the short syntax:
 class C(T) {...} Also, "public" is default in D. I don't know if that holds
true
 for inheritance, though. If it's also by default there, then it wouldn't be
 neede in your example.

Gotcha I'll migrate to the D way for templates, then, but I'm keeping with my explicit access declarations. It aids readability, discoverability and instrumentability (and ilityability <G>), and costs but a couple of axon firings.
Jul 31 2004
prev sibling parent reply "Carlos Santander B." <carlos8294 msn.com> writes:
"Matthew" <admin.hat stlsoft.dot.org> escribió en el mensaje
news:ceaeid$18ir$1 digitaldaemon.com
|
| [very interesting read]
|

I have used STL only to play. Once I read TC++PL (I wanted to give C++ a try:
sorry, no time). I've done even less with Java's Collections API. I've written
somethings with D's foreach(). So, while I don't have a large knowledge about
them, I understand they're very different approaches to the same issue, and to
see you're trying to do all of that, is just way over my head.

There's a question coming: could it be possible that you're trying to achieve
too much? I know it's a bit late for it, but do you think it could've been
easier if you had done first this, then that, etc.? Just asking.

About your boxing module, did you know Andy wrote that some time ago? I haven't
tested it, but I know it exists.

About your first litany, I agree with you. Sometimes something's working, and
you add just one tiny correct thing, and the compiler gets lost. Sometimes just
reorganizing modules makes everything stop working. And it's certainly
impossible to reproduce it.

Besides that, I just can say don't give up. DTL looks very promising and I don't
think any of us would like to lose it (and you) at this point. And release
something. I'm not saying I'm going to help (I'm not saying I'm not, either),
but, who knows, with luck everyone will stop what they're doing and check DTL
and suggest things.

-----------------------
Carlos Santander Bernal
Jul 29 2004
next sibling parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Carlos Santander B." <carlos8294 msn.com> wrote in message
news:cec3is$1ug7$1 digitaldaemon.com...
 "Matthew" <admin.hat stlsoft.dot.org> escribió en el mensaje
 news:ceaeid$18ir$1 digitaldaemon.com
 |
 | [very interesting read]
 |

 I have used STL only to play. Once I read TC++PL (I wanted to give C++ a try:
 sorry, no time). I've done even less with Java's Collections API. I've written
 somethings with D's foreach(). So, while I don't have a large knowledge about
 them, I understand they're very different approaches to the same issue, and to
 see you're trying to do all of that, is just way over my head.

 There's a question coming: could it be possible that you're trying to achieve
 too much?

It's certainly likely that I've tried to achieve too much in the first instance. But, no, I do not believe the DTL "vision" is overblown. In fact, I think it's really neat, easily explained, and self-contained. (But then I would think that, I suppose)
 I know it's a bit late for it, but do you think it could've been
 easier if you had done first this, then that, etc.? Just asking.

Most certainly.
 About your boxing module, did you know Andy wrote that some time ago? I haven't
 tested it, but I know it exists.

No. Where is it? How to I get it? I'd like to get a look at that. btw, do you mean Andy Friesen? Andy, want to knock heads together on this? Mine was just a stab to get me what I needed, and was not intended to be the all-things-for-all-men (or women!) approach. However, I thought it might serve as a base for that.
 About your first litany, I agree with you. Sometimes something's working, and
 you add just one tiny correct thing, and the compiler gets lost. Sometimes just
 reorganizing modules makes everything stop working. And it's certainly
 impossible to reproduce it.

Hurrah! So it's not just me, then.
 Besides that, I just can say don't give up. DTL looks very promising and I
don't
 think any of us would like to lose it (and you) at this point.

Thanks. I confess I've been skirting the throwing-the-towel-in point over the last couple of weeks. Thankfully, the gritty Yorkshireman in me jumped out and gave me a kick up the arse. :)
 And release
 something. I'm not saying I'm going to help (I'm not saying I'm not, either),
 but, who knows, with luck everyone will stop what they're doing and check DTL
 and suggest things.

Ok, arm-twisters all. I shall release 0.1 this weekend. It will only have type 1 ("foreach") enumeration capabilities for each container, although some of the other enumeration types will be available for some of the containers. Then we can take it from there. (btw, the actual "containment" implementations were all done in a couple of days in March, so they may not be optimal. That doesn't matter of course, but I just don't want anyone to think that any aspects of the lib are production-ready yet.)
Jul 29 2004
next sibling parent reply "Carlos Santander B." <carlos8294 msn.com> writes:
"Matthew" <admin stlsoft.dot.dot.dot.dot.org> escribió en el mensaje
news:cec85j$204l$1 digitaldaemon.com
| "Carlos Santander B." <carlos8294 msn.com> wrote in message
| news:cec3is$1ug7$1 digitaldaemon.com...
|| There's a question coming: could it be possible that you're trying to achieve
|| too much?
|
| It's certainly likely that I've tried to achieve too much in the first
instance.
|
| But, no, I do not believe the DTL "vision" is overblown. In fact, I think it's
| really neat, easily explained, and self-contained. (But then I would think
that, I
| suppose)

I meant that in the context of what followed.

|
|| I know it's a bit late for it, but do you think it could've been
|| easier if you had done first this, then that, etc.? Just asking.
|
| Most certainly.
|
|| About your boxing module, did you know Andy wrote that some time ago? I
haven't
|| tested it, but I know it exists.
|
| No. Where is it? How to I get it? I'd like to get a look at that.
|
| btw, do you mean Andy Friesen? Andy, want to knock heads together on this?
|
| Mine was just a stab to get me what I needed, and was not intended to be the
| all-things-for-all-men (or women!) approach. However, I thought it might serve
as
| a base for that.

Yes, Andy Friesen indeed. It's part of his apropos library, available at
http://andy.tadan.us/d/.

|
|| About your first litany, I agree with you. Sometimes something's working, and
|| you add just one tiny correct thing, and the compiler gets lost. Sometimes
just
|| reorganizing modules makes everything stop working. And it's certainly
|| impossible to reproduce it.
|
| Hurrah! So it's not just me, then.
|
|| Besides that, I just can say don't give up. DTL looks very promising and I
don't
|| think any of us would like to lose it (and you) at this point.
|
| Thanks. I confess I've been skirting the throwing-the-towel-in point over the
last
| couple of weeks. Thankfully, the gritty Yorkshireman in me jumped out and gave
me
| a kick up the arse. :)

No problem.

|
|
|| And release
|| something. I'm not saying I'm going to help (I'm not saying I'm not, either),
|| but, who knows, with luck everyone will stop what they're doing and check DTL
|| and suggest things.
|
| Ok, arm-twisters all. I shall release 0.1 this weekend. It will only have type
1
| ("foreach") enumeration capabilities for each container, although some of the
| other enumeration types will be available for some of the containers. Then we
can
| take it from there.
|
| (btw, the actual "containment" implementations were all done in a couple of
days
| in March, so they may not be optimal. That doesn't matter of course, but I
just
| don't want anyone to think that any aspects of the lib are production-ready
yet.)

I think most of us will be happy with that.

-----------------------
Carlos Santander Bernal
Jul 29 2004
parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"Carlos Santander B." <carlos8294 msn.com> wrote in message
news:cecbh5$21hg$1 digitaldaemon.com...
 "Matthew" <admin stlsoft.dot.dot.dot.dot.org> escribió en el mensaje
 news:cec85j$204l$1 digitaldaemon.com
 | "Carlos Santander B." <carlos8294 msn.com> wrote in message
 | news:cec3is$1ug7$1 digitaldaemon.com...
 || There's a question coming: could it be possible that you're trying to
achieve
 || too much?
 |
 | It's certainly likely that I've tried to achieve too much in the first
 instance.
 |
 | But, no, I do not believe the DTL "vision" is overblown. In fact, I think
it's
 | really neat, easily explained, and self-contained. (But then I would think
 that, I
 | suppose)

 I meant that in the context of what followed.

No worries. :)
 |
 || I know it's a bit late for it, but do you think it could've been
 || easier if you had done first this, then that, etc.? Just asking.
 |
 | Most certainly.
 |
 || About your boxing module, did you know Andy wrote that some time ago? I
 haven't
 || tested it, but I know it exists.
 |
 | No. Where is it? How to I get it? I'd like to get a look at that.
 |
 | btw, do you mean Andy Friesen? Andy, want to knock heads together on this?
 |
 | Mine was just a stab to get me what I needed, and was not intended to be the
 | all-things-for-all-men (or women!) approach. However, I thought it might
serve
 as
 | a base for that.

 Yes, Andy Friesen indeed. It's part of his apropos library, available at
 http://andy.tadan.us/d/.

 |
 || About your first litany, I agree with you. Sometimes something's working,
and
 || you add just one tiny correct thing, and the compiler gets lost. Sometimes
 just
 || reorganizing modules makes everything stop working. And it's certainly
 || impossible to reproduce it.
 |
 | Hurrah! So it's not just me, then.
 |
 || Besides that, I just can say don't give up. DTL looks very promising and I
 don't
 || think any of us would like to lose it (and you) at this point.
 |
 | Thanks. I confess I've been skirting the throwing-the-towel-in point over the
 last
 | couple of weeks. Thankfully, the gritty Yorkshireman in me jumped out and
gave
 me
 | a kick up the arse. :)

 No problem.

 |
 |
 || And release
 || something. I'm not saying I'm going to help (I'm not saying I'm not,
either),
 || but, who knows, with luck everyone will stop what they're doing and check
DTL
 || and suggest things.
 |
 | Ok, arm-twisters all. I shall release 0.1 this weekend. It will only have
type
 1
 | ("foreach") enumeration capabilities for each container, although some of the
 | other enumeration types will be available for some of the containers. Then we
 can
 | take it from there.
 |
 | (btw, the actual "containment" implementations were all done in a couple of
 days
 | in March, so they may not be optimal. That doesn't matter of course, but I
 just
 | don't want anyone to think that any aspects of the lib are production-ready
 yet.)

 I think most of us will be happy with that.

Hope so.
Jul 29 2004
prev sibling parent reply Andy Friesen <andy ikagames.com> writes:
Matthew wrote:

 No. Where is it? How to I get it? I'd like to get a look at that.
 
 btw, do you mean Andy Friesen? Andy, want to knock heads together on this?
 
 Mine was just a stab to get me what I needed, and was not intended to be the
all-things-for-all-men (or women!)
 approach. However, I thought it might serve as a base for that.

Yeah, that's me. I put a copy online at <http://andy.tadan.us/d/variant.d> It's about as simple as it can get. :) -- andy
Jul 29 2004
parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"Andy Friesen" <andy ikagames.com> wrote in message
news:cece6h$22hj$1 digitaldaemon.com...
 Matthew wrote:

 No. Where is it? How to I get it? I'd like to get a look at that.

 btw, do you mean Andy Friesen? Andy, want to knock heads together on this?

 Mine was just a stab to get me what I needed, and was not intended to be the
all-things-for-all-men (or women!)
 approach. However, I thought it might serve as a base for that.

Yeah, that's me. I put a copy online at <http://andy.tadan.us/d/variant.d> It's about as simple as it can get. :)

It seems to be a different approach to mine, and not really conflicting/redundant. My Box!() template (included) is almost offensively simple, it merely turns a built-in type into an Object-derived type, as in: alias Box!(int) BoxedInt_t; int i = 10; BoxedInt_t bi = new BoxedInt_t(i); i = bi.value; if(bi == i) {} if(bi != i) {} if(bi < i) {} (Actually, not all of that is yet implemented, but it will be.)
Jul 30 2004
prev sibling parent "Walter" <newshound digitalmars.com> writes:
"Carlos Santander B." <carlos8294 msn.com> wrote in message
news:cec3is$1ug7$1 digitaldaemon.com...
 Sometimes just
 reorganizing modules makes everything stop working. And it's certainly
 impossible to reproduce it.

Compilers are very deterministic. If you have a series of steps that "doesn't work", it's reproducible.
Jul 29 2004