www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: Developing a plan for D2.0: Getting everything on the table

reply Jason House <jason.james.house gmail.com> writes:
Other, less technical items:
• A clear and "finalized" spec. If it isn't implemented, it should be yanked
(or clearly marked as pending)
• A plan for library support. Not just Tango, but also Phobos. D1 Phobos could
not evolve.

Don Wrote:

 A lot of frustration has been expressed on the newgroup about lack of a 
 clear public plan for D2.0. I don't think we're in a position to create 
 a road-map. But, let's at least agree on which countries we'll probably 
 visit before we reach our final destination <g>.
 
 Everyone knows there are a multitude of significant bugs in Bugzilla, 
 and most people have their pet list of minor language warts they hope 
 will be removed. But there's also some earthquake issues that have huge 
 implications. It's very disconcerting when some of them are introduced 
 in a casual manner. I think it would reduce a lot of frustation in the 
 community if we compiled an official list of the major ones. Here's a 
 few I came up with:
 
 - Multithreading (I): Will Bartosz's proposal be accepted (in some form)?
 - Multithreading (II): Will some form of message parsing be included?
 - Operator overloading. "completely redone" (?)
 - opImplicitCast
 - is T[new] still going to happen?
 - Phobos I/O -- Andrei has stated that he wants to completely rewrite it.
 - Unimplemented features -- safe D, contract inheritance.
 - Andrei once said that he wants to get rid of new (!)
 - The Tango license issue needs to be sorted to the extent that Andrei 
 and Walter feel they can safely look at the Tango code; OR we can decide 
 that's not going to happen, and change the strategy for the Tango/Phobos 
 relationship.
 
 The stuff on this list will either be implemented, or dropped. New 
 things could be added to the list. But we can gauge our progress towards 
 D2.0 by how rapidly the list shrinks with time.
 
 Which other major issues have I missed? Things which, if they happen, 
 will probably require major spec changes, major library redesign, or 
 break large amounts of code. Let's get everything on the table.

Jul 14 2009
parent reply Benji Smith <dlanguage benjismith.net> writes:
Jason House wrote:
 Other, less technical items:
 • A clear and "finalized" spec. If it isn't implemented, it should be yanked
(or clearly marked as pending)
 • A plan for library support. Not just Tango, but also Phobos. D1 Phobos could
not evolve.

In D1, I enthusiastically used Tango. I haven't used D2 yet (because all my code is heavily tied to the Tango libs), but I suspect that when D2 is finalized, I'll port everything over to Phobos. I've read all the Phobos2 development discussions here (most notably the range discussions), but what about the feature disparities between the two libraries. What types of functionality are currently present in Tango but absent in Phobos? Maybe if Andrei put together a list of missing Phobos functionality, we could get people from the community to flesh out the libs. For example, I have a JSON parser implementation that I'd be happy to contribute. --benji
Jul 22 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Benji Smith wrote:
 Jason House wrote:
 Other, less technical items:
 • A clear and "finalized" spec. If it isn't implemented, it should be 
 yanked (or clearly marked as pending)
 • A plan for library support. Not just Tango, but also Phobos. D1 
 Phobos could not evolve.

In D1, I enthusiastically used Tango. I haven't used D2 yet (because all my code is heavily tied to the Tango libs), but I suspect that when D2 is finalized, I'll port everything over to Phobos. I've read all the Phobos2 development discussions here (most notably the range discussions), but what about the feature disparities between the two libraries. What types of functionality are currently present in Tango but absent in Phobos? Maybe if Andrei put together a list of missing Phobos functionality, we could get people from the community to flesh out the libs.

I think we'd need at a minimum: * better networking library * containers * just a little linear algebra basics (i.e. well-defined storage that would allow us to interface with high-performance libraries) * at least some essentials for compile-time introspection (Shin, I'm still working on integrating WhiteHole and BlackHole; I want to expand your support function into a better fleshed and more general functionality) * a few functions here and there, e.g. readf * a complete rewrite of std.xml which is currently so slow it's not even funny * of course last but not least concurrency support * some more generic types a la std.typecons * flesh out std.complex which is to supplant the dying complex built-in type
 For example, I have a JSON parser implementation that I'd be happy to 
 contribute.

That would be great. In general, it would be awesome to gather more contributions from the community. There's a thirst to contribute and we'll be glad to involve this group into some serious design e.g. for concurrency support, as well as accept code for functionality that belongs to the standard library. In the bulleted list above there are many mini-projects that are confined enough to be done by one willing individual in a relatively short time. Andrei
Jul 22 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:
 On Wed, 22 Jul 2009 21:55:54 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 
 I think we'd need at a minimum:

what are your opinions on the I/O subsystem? I think a lot of performance/features could be gained by using D-based buffered I/O instead of the C standard lib. Tango is pretty much a testament to that...

Better speed is always nice, so it would be great to see some work in that direction. What are the specific shortcomings that make using stdio unrecommended? Andrei
Jul 22 2009
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jesse Phillips wrote:
 On Wed, 22 Jul 2009 20:55:54 -0500, Andrei Alexandrescu wrote:
 
 I think we'd need at a minimum:

 * better networking library

 * containers

 * just a little linear algebra basics (i.e. well-defined storage that
 would allow us to interface with high-performance libraries)

 * at least some essentials for compile-time introspection (Shin, I'm
 still working on integrating WhiteHole and BlackHole; I want to expand
 your support function into a better fleshed and more general
 functionality)

 * a few functions here and there, e.g. readf

 * a complete rewrite of std.xml which is currently so slow it's not even
 funny

 * of course last but not least concurrency support

 * some more generic types a la std.typecons

 * flesh out std.complex which is to supplant the dying complex built-in
 type


 Andrei

These points were added to: http://www.prowiki.org/wiki4d/wiki.cgi?PhobosToDo Where they shall sit a long horrible life :P

I suggest anyone who wants to do something for D to focus on the installation aspect. It's a relatively well-defined project of high importance and high visibility. We simply can't afford to introduce D to the world without a state-of-the-art installation procedure. Andrei
Jul 22 2009
next sibling parent =?UTF-8?B?QW5kZXJzIEYgQmrDtnJrbHVuZA==?= <afb algonet.se> writes:
Andrei Alexandrescu wrote:
 I suggest anyone who wants to do something for D to focus on the 
 installation aspect. It's a relatively well-defined project of high 
 importance and high visibility. We simply can't afford to introduce D to 
 the world without a state-of-the-art installation procedure.

I made some installers for the DMD 1.00 introduction, but back then the .zip (or the missing .tgz) were still the preferred procedure... http://www.algonet.se/~afb/d/dmd-setup.html for Windows http://www.algonet.se/~afb/d/rpm-setup.html for Linux So the installers were only used for the portable D compiler (gdc) in the complementary gdcwin/gdcgnu/gcmac projects. The dgcc upstream always used .tar.bz2, similar to how dmd used .zip for the releases. But with proper documentation, it wasn't *that* much extra hassle... Going through the installation wizards is more "familiar", but probably not less work than going through the install steps with unzip, cp, chmod and $PATH and whatnot, with the contributions at: http://www.prowiki.org/wiki4d/wiki.cgi?D__Tutorial/InstallingDCompiler I like having installers myself, but it seemed like install problems were mostly given as "yet another excuse" to not try the D language. Even with the dmd/gdc installers, you'd get the "where's the IDE" or "where's the GUI" so it's mostly about documentation and presentation ? I won't do any dmd2/ldc installers myself I think, but will keep the dmd1/gdc installers up for as long as there seems to be interest... Should probably update the SF web pages with the new information on the development of D, since currently they all date back to 2008. --anders
Jul 23 2009
prev sibling parent reply Jason House <jason.james.house gmail.com> writes:
Andrei Alexandrescu Wrote:

 I suggest anyone who wants to do something for D to focus on the 
 installation aspect. It's a relatively well-defined project of high 
 importance and high visibility. We simply can't afford to introduce D to 
 the world without a state-of-the-art installation procedure.
 
 
 Andrei

Not everyone is capable of effeciently helping in that area. Will there ever be an official list of things like this? Searching the mailing list is not an effective way of finding what you/Walter consider to be valuable additions. A list allows a motivated contributor to quickly find something that interests them. Having only one item also makes people think that somebody else is probably doing it and they don't need to help.
Jul 23 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jason House wrote:
 Andrei Alexandrescu Wrote:
 
 I suggest anyone who wants to do something for D to focus on the 
 installation aspect. It's a relatively well-defined project of high 
 importance and high visibility. We simply can't afford to introduce D to 
 the world without a state-of-the-art installation procedure.


 Andrei

Not everyone is capable of effeciently helping in that area. Will there ever be an official list of things like this? Searching the mailing list is not an effective way of finding what you/Walter consider to be valuable additions. A list allows a motivated contributor to quickly find something that interests them. Having only one item also makes people think that somebody else is probably doing it and they don't need to help.

You may want to bookmark http://www.prowiki.org/wiki4d/wiki.cgi?PhobosToDo Andrei
Jul 23 2009
prev sibling next sibling parent reply Benji Smith <dlanguage benjismith.net> writes:
Andrei Alexandrescu wrote:
 Benji Smith wrote:
 Maybe if Andrei put together a list of 
 missing Phobos functionality, we could get people from the community 
 to flesh out the libs.

I think we'd need at a minimum:

<snip>
 That would be great. In general, it would be awesome to gather more 
 contributions from the community. There's a thirst to contribute and 
 we'll be glad to involve this group into some serious design e.g. for 
 concurrency support, as well as accept code for functionality that 
 belongs to the standard library.
 
 In the bulleted list above there are many mini-projects that are 
 confined enough to be done by one willing individual in a relatively 
 short time.

Are there contributor guidelines somewhere? For example, should the author of a container library prefer classes or structs? Should other (non-container) modules accept container classes as arguments? Or only container interfaces (if there are any such things) or just ranges? Is it appropriate to use an empty struct purely as a namespace for the introduction of free functions? Or should free functions be placed at the module level? Is it appropriate to define multiple classes, structs, templates, etc within a single module? What considerations should inform the decision regarding the placement of module boundaries? What constitutes appropriate/inappropriate usage of opCall? Anyhoo... Point being, Phobos_1 was a hodgepodge of different conventions and styles. Tango_1 was considerably better, in terms of stylistic uniformity. But it used a very different set of idioms than Phobos_1 (lots of predicate functions, "sink" delegates, etc). Probably any author contributing code to Phobos_2 should spend a little time getting up to speed with the preferred idioms before writing code. I suspect that my humble little JSON parser uses styles and idioms that would clash with the majority of Phobos_2 (since my programming pedigree comes from Java, C#, JavaScript, and Perl much moreso than C or C++). --benji
Jul 26 2009
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-07-26 23:36:06 -0400, Benji Smith <dlanguage benjismith.net> said:

 Andrei Alexandrescu wrote:
 Benji Smith wrote:
 Maybe if Andrei put together a list of
 missing Phobos functionality, we could get people from the community to 
 flesh out the libs.

I think we'd need at a minimum:

<snip>
 That would be great. In general, it would be awesome to gather more 
 contributions from the community. There's a thirst to contribute and 
 we'll be glad to involve this group into some serious design e.g. for 
 concurrency support, as well as accept code for functionality that 
 belongs to the standard library.
 
 In the bulleted list above there are many mini-projects that are 
 confined enough to be done by one willing individual in a relatively 
 short time.

Are there contributor guidelines somewhere?

Funny. I started to write some guidelines a few months ago. Those were guidelines about how to name things consistently: like types should be camel-cased nouns possibly prefixed with adjectives starting with an uppercase; variables, getters, and setters should be camel-cased nouns starting with a lowercase; functions that perform an action should contain with a verb, short names should be avoided if they aren't meaningful, etc. Then I found that most of Phobos would have to change to fit my guidelines, and I couldn't come with anything sensible that would fit half of Phobos, so I gave up. If someone's insterested, I can post them somewhere. Unfortunatly, I wonder if this lack of guidelines is slowing contributions to Phobos, with everyone who aims at good integration wasting their time trying to figure out the rules. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jul 27 2009
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Benji Smith wrote:
 Andrei Alexandrescu wrote:
 Benji Smith wrote:
 Maybe if Andrei put together a list of
 missing Phobos functionality, we could get people from the community 
 to flesh out the libs.

I think we'd need at a minimum:

<snip>
 That would be great. In general, it would be awesome to gather more 
 contributions from the community. There's a thirst to contribute and 
 we'll be glad to involve this group into some serious design e.g. for 
 concurrency support, as well as accept code for functionality that 
 belongs to the standard library.

 In the bulleted list above there are many mini-projects that are 
 confined enough to be done by one willing individual in a relatively 
 short time.

Are there contributor guidelines somewhere?

Not at the moment.
 For example, should the author of a container library prefer classes or 
 structs?

I've been struggling with this forever. I don't know. I don't even know whether reference or value semantics are best for containers. I don't know whether abstract container interfaces and container-independent code are a net win; experience with STL seems to say "don't" and experience with Java seems to say "ho-hum".
 Should other (non-container) modules accept container classes 
 as arguments? Or only container interfaces (if there are any such 
 things) or just ranges?

I think ranges should be preferred wherever applicable.
 Is it appropriate to use an empty struct purely as a namespace for the 
 introduction of free functions? Or should free functions be placed at 
 the module level?

I think module-level functions are best. If there are lookup issues due to that, we need to fix the language. D is conceived to allow proper coexistence of functions in different modules without forcing the user to take too many precautions.
 Is it appropriate to define multiple classes, structs, templates, etc 
 within a single module? What considerations should inform the decision 
 regarding the placement of module boundaries?

I think it's appropriate because many pieces of functionality come as a bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails.
 What constitutes appropriate/inappropriate usage of opCall?

What examples do you have in mind?
 Anyhoo...
 
 Point being, Phobos_1 was a hodgepodge of different conventions and 
 styles. Tango_1 was considerably better, in terms of stylistic 
 uniformity. But it used a very different set of idioms than Phobos_1 
 (lots of predicate functions, "sink" delegates, etc). Probably any 
 author contributing code to Phobos_2 should spend a little time getting 
 up to speed with the preferred idioms before writing code.

I agree. I am obviously subjective, but here are a few modules in phobos2 that seem to me close to the desired look and feel: * std.algorithm is a large collection of functions mostly operating on one abstraction (ranges). If you know how to use one algorithm, you probably know how to use all others. Defining ranges that std.algorithm can work with is also a breeze. * std.getopt is a very effective module. Although it does a fair amount of stuff, it has a simple interface that can be learned in minutes. I haven't seen an options parsing library that has the same power/ease ratio. Its usage of static types to infer what you want to do is a good showcase for the power of D templates and variadics. * std.conv is simple (in interface), convenient, and general. * std.range could use some maturing. I'm currently not 100% pleased about it. * std.file looks a bit quaint (e.g. no classes) but I'm comfortable around it. It does what it's supposed to do and does it well. * std.numeric is an example of a library that has a mix of functions and objects collected under an umbrella topic. I'm ok with it, but at some point I suspect I'd start worrying about its growth. Same about e.g. std.typecons. * std.md5 is an example on why it's not that advisable to just copy a C interface into D. (This is a negative example.) * std.random is a simple, extensible design, and has convenience functions (such as uniform) that are, well, darn convenient. I see it growing easily and nicely.
 I suspect that my humble little JSON parser uses styles and idioms that 
 would clash with the majority of Phobos_2 (since my programming pedigree 
 comes from Java, C#, JavaScript, and Perl much moreso than C or C++).

Phobos2 doesn't have (m)any hierarchies not because Walter or I don't use them or like them. It's just that the particular stuff that's in Phobos right now is at an advantage by using other design means. But if you come with a good hierarchy implementing a piece of functionality, great. Andrei
Jul 27 2009
next sibling parent reply yigal chripun <yigal100 gmail.com> writes:
Andrei Alexandrescu Wrote:

 Is it appropriate to define multiple classes, structs, templates, etc 
 within a single module? What considerations should inform the decision 
 regarding the placement of module boundaries?

I think it's appropriate because many pieces of functionality come as a bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails. Andrei

The rule of thumb above is good in theory but in practice such a given bundle of functionality can produce a 30_000 line file. while all those lines should be "bundled" conceptually together, D's restriction of module == file makes for a huge file that is a hassle to work with. yes, a "module" should represent a conceptual bundle of functionallity but forcing this into one huge file is wrong. -- Yigal
Jul 27 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
yigal chripun wrote:
 Andrei Alexandrescu Wrote:
 
 Is it appropriate to define multiple classes, structs, templates, etc 
 within a single module? What considerations should inform the decision 
 regarding the placement of module boundaries?

bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails. Andrei

The rule of thumb above is good in theory but in practice such a given bundle of functionality can produce a 30_000 line file. while all those lines should be "bundled" conceptually together, D's restriction of module == file makes for a huge file that is a hassle to work with. yes, a "module" should represent a conceptual bundle of functionallity but forcing this into one huge file is wrong.

I think public import can help with that. Andrei
Jul 27 2009
parent reply yigal chripun <yigal100 gmail.com> writes:
Andrei Alexandrescu Wrote:

 yigal chripun wrote:
 Andrei Alexandrescu Wrote:
 
 Is it appropriate to define multiple classes, structs, templates, etc 
 within a single module? What considerations should inform the decision 
 regarding the placement of module boundaries?

bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails. Andrei

The rule of thumb above is good in theory but in practice such a given bundle of functionality can produce a 30_000 line file. while all those lines should be "bundled" conceptually together, D's restriction of module == file makes for a huge file that is a hassle to work with. yes, a "module" should represent a conceptual bundle of functionallity but forcing this into one huge file is wrong.

I think public import can help with that. Andrei

That is just a work-around and not a full solution. this is a weak spot in D which I'd like to see fixed. I've already suggested in the past some ideas on how this can be solved. we need a reasonable tradeoff between the namespaces design of C# which is completely seperated from physical layout (filesystem) and the packages design of Java which is tightly coupled with the filesystem.
Jul 27 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
yigal chripun wrote:
 Andrei Alexandrescu Wrote:
 
 yigal chripun wrote:
 Andrei Alexandrescu Wrote:
 
 Is it appropriate to define multiple classes, structs,
 templates, etc within a single module? What considerations
 should inform the decision regarding the placement of module
 boundaries?

come as a bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails. Andrei

given bundle of functionality can produce a 30_000 line file. while all those lines should be "bundled" conceptually together, D's restriction of module == file makes for a huge file that is a hassle to work with. yes, a "module" should represent a conceptual bundle of functionallity but forcing this into one huge file is wrong.

Andrei

That is just a work-around and not a full solution.

Why?
 this is a weak
 spot in D which I'd like to see fixed.

Why is it a weak spot? What problems mar its advantages? Andrei
Jul 27 2009
parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Andrei Alexandrescu wrote:
 yigal chripun wrote:
 Andrei Alexandrescu Wrote:

 yigal chripun wrote:
 Andrei Alexandrescu Wrote:

 Is it appropriate to define multiple classes, structs,
 templates, etc within a single module? What considerations
 should inform the decision regarding the placement of module
 boundaries?

come as a bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails. Andrei

given bundle of functionality can produce a 30_000 line file. while all those lines should be "bundled" conceptually together, D's restriction of module == file makes for a huge file that is a hassle to work with. yes, a "module" should represent a conceptual bundle of functionallity but forcing this into one huge file is wrong.

Andrei

That is just a work-around and not a full solution.

Why?
 this is a weak
 spot in D which I'd like to see fixed.

Why is it a weak spot? What problems mar its advantages?

I think the approach with public imports should work in principle (although would probably require making a package for such 'module bundles' to have package-level field visibility, since private members would not be accessible across modules), but currently the DMD frontend fails at circular dependencies making this approach impossible. -- Tomasz Stachowiak http://h3.team0xf.com/ h3/h3r3tic on #D freenode
Jul 27 2009
prev sibling next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el 27 de julio a las 07:59 me escribiste:
For example, should the author of a container library prefer classes or 
structs?

I've been struggling with this forever. I don't know. I don't even know whether reference or value semantics are best for containers. I don't know whether abstract container interfaces and container-independent code are a net win; experience with STL seems to say "don't" and experience with Java seems to say "ho-hum".

About values vs. reference semantics, I think reference is the best. Containers usually are big, and you don't want to copy them arround. I find myself using lots of references to pass containers arround in C++ and almost never use the copy() method of Python containers, so based on *my* experience, I'd say that reference as the default is the best approach. -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------------- Seducimos a una minita con el objetivo de llevarla a nuestro hogar; mรกs que seducirla la amenazamos con una botella. -- Sidharta Kiwi
Jul 27 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Andrei Alexandrescu, el 27 de julio a las 07:59 me escribiste:
 For example, should the author of a container library prefer classes or 
 structs?

whether reference or value semantics are best for containers. I don't know whether abstract container interfaces and container-independent code are a net win; experience with STL seems to say "don't" and experience with Java seems to say "ho-hum".

About values vs. reference semantics, I think reference is the best. Containers usually are big, and you don't want to copy them arround. I find myself using lots of references to pass containers arround in C++ and almost never use the copy() method of Python containers, so based on *my* experience, I'd say that reference as the default is the best approach.

Sounds convincing. Andrei
Jul 27 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:
 On Mon, Jul 27, 2009 at 9:54 AM, Bill Baxter<wbaxter gmail.com> wrote:
 On Mon, Jul 27, 2009 at 7:48 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:
 Leandro Lucarella wrote:
 Andrei Alexandrescu, el 27 de julio a las 07:59 me escribiste:
 For example, should the author of a container library prefer classes or
 structs?

whether reference or value semantics are best for containers. I don't know whether abstract container interfaces and container-independent code are a net win; experience with STL seems to say "don't" and experience with Java seems to say "ho-hum".

Containers usually are big, and you don't want to copy them arround. I find myself using lots of references to pass containers arround in C++ and almost never use the copy() method of Python containers, so based on *my* experience, I'd say that reference as the default is the best approach.


want is value semantics. I've just finished recently refactoring some C++ code that wasn't designed with copying in mind. Changing all the "float* data; int data_length;" members into std::vectors did the trick. The data they contained wasn't particularly large, and there was no real need to introduce the complexity that sharing it between copied instances would have created.

So if I didn't make it clear -- my opinion is that whether containers should be by value or by ref depends on the circumstances and that choice should be left to the programmer. Thus the question becomes what's the easiest way to provide both? Three options I see: 1) Write ref classes and wrap them in structs 2) Write value structs and wrap them in classes 3) Write template mixins and mix them into both struct and class shells. I think given the existence of "alias this" for structs, 1 is actually the best-supported paradigm. I think with that you should be able to value-ize a by-ref container class just by writing a post-blitter to copy the contents. --bb

These thoughts are very much aligned with mine. Thanks Bill. Andrei
Jul 27 2009
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:
 The requirement to add the post-blit is a little annoying.  If a
 ValueContainer is made fundamental and it is built initially out of
 components with value semantics, then no explicit post-blit will be
 necessary.  However if you go the other way and start with a
 RefContainer then a post-blit will be necessary when you implement the
 ValueContainer based on it.   So to me it seems a little better to go
 with the ValueContainer as the base implementation.

I think it's a matter of the most frequent use. I used to be a staunch supporter of values for containers, until Walter revealed to me that in STL you have value containers to constantly worry about adding the "&" everywhere. Implicitly copying containers is 95% of cases an error in C++ code. Witness all that advice about pass by reference etc. So why make that the default, one might ask? Better go with the flow and make the default case the easy case (reference), and if someone does want a value, have them write Value!Container. Andrei
Jul 27 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el 27 de julio a las 12:58 me escribiste:
 Bill Baxter wrote:
The requirement to add the post-blit is a little annoying.  If a
ValueContainer is made fundamental and it is built initially out of
components with value semantics, then no explicit post-blit will be
necessary.  However if you go the other way and start with a
RefContainer then a post-blit will be necessary when you implement the
ValueContainer based on it.   So to me it seems a little better to go
with the ValueContainer as the base implementation.

I think it's a matter of the most frequent use. I used to be a staunch supporter of values for containers, until Walter revealed to me that in STL you have value containers to constantly worry about adding the "&" everywhere. Implicitly copying containers is 95% of cases an error in C++ code. Witness all that advice about pass by reference etc. So why make that the default, one might ask? Better go with the flow and make the default case the easy case (reference), and if someone does want a value, have them write Value!Container.

Another point for ref containers is that they allow to have a class hierarchy of containers. I guess you can have it too using compile-time "concepts", like ranges, but being able to do that with interfaces could be nice (and you can still have the compile-time with classes too). I don't know if the interfaces approach would be useful though... -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------------- We're rotten fruit We're damaged goods What the hell, we've got nothing more to lose One gust and we will probably crumble We're backdrifters
Jul 27 2009
prev sibling parent Brad Roberts <braddr bellevue.puremagic.com> writes:
On Mon, 27 Jul 2009, Andrei Alexandrescu wrote:

 Bill Baxter wrote:
 The requirement to add the post-blit is a little annoying.  If a
 ValueContainer is made fundamental and it is built initially out of
 components with value semantics, then no explicit post-blit will be
 necessary.  However if you go the other way and start with a
 RefContainer then a post-blit will be necessary when you implement the
 ValueContainer based on it.   So to me it seems a little better to go
 with the ValueContainer as the base implementation.

I think it's a matter of the most frequent use. I used to be a staunch supporter of values for containers, until Walter revealed to me that in STL you have value containers to constantly worry about adding the "&" everywhere. Implicitly copying containers is 95% of cases an error in C++ code. Witness all that advice about pass by reference etc. So why make that the default, one might ask? Better go with the flow and make the default case the easy case (reference), and if someone does want a value, have them write Value!Container. Andrei

There's two layers to talk about, the container itself and the elements it contains. I agree that the container itself should be ref given the above logic (which matches my experience). The less obvious one, imho, is the default of the contained. Again, c++ uses value containership, but an awful lot of the time people (including myself) use essentially a ref (by way of a smart pointer). Often it's due to weak ownership semantics, and thus iffy design, but it is what it is. Later, Brad
Jul 27 2009
prev sibling parent Rainer Deyke <rainerd eldwood.com> writes:
Leandro Lucarella wrote:
 About values vs. reference semantics, I think reference is the best.
 Containers usually are big, and you don't want to copy them arround.
 I find myself using lots of references to pass containers arround in C++
 and almost never use the copy() method of Python containers, so based on
 *my* experience, I'd say that reference as the default is the best
 approach.

I don't think the choice of reference vs value semantics has *anything* to do with how often something is copied. To me, the relevant questions are: - Do you have strict ownership relationships? (Usually you do.) - Do you want to avoid long-distance bugs? (Yes you do, always.) Orthogonal question: does copying *ever* make sense: - If yes, provide a copy operation. - If no, prevent copying. -- Rainer Deyke - rainerd eldwood.com
Jul 27 2009
prev sibling parent reply Sergey Gromov <snake.scaly gmail.com> writes:
Mon, 27 Jul 2009 07:59:40 -0500, Andrei Alexandrescu wrote:

 Is it appropriate to define multiple classes, structs, templates, etc 
 within a single module? What considerations should inform the decision 
 regarding the placement of module boundaries?

I think it's appropriate because many pieces of functionality come as a bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails.

That's the problem. On one hand, it's desirable to see a module as a functionality bundle. On the other hand, module is the smallest available unit of encapsulation. That is, if you have a class and change its private implementation, this may affect *anything* in the same module. Hence Tango's hundreds of modules in dozens of packages, I think. It also adds to the problem that most people familiar with OO paradigm consider classes to be such encapsulation units. Surprizes are inevitable.
Jul 28 2009
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Sergey Gromov wrote:
 Mon, 27 Jul 2009 07:59:40 -0500, Andrei Alexandrescu wrote:
 
 Is it appropriate to define multiple classes, structs, templates, etc 
 within a single module? What considerations should inform the decision 
 regarding the placement of module boundaries?

bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails.

That's the problem. On one hand, it's desirable to see a module as a functionality bundle. On the other hand, module is the smallest available unit of encapsulation. That is, if you have a class and change its private implementation, this may affect *anything* in the same module. Hence Tango's hundreds of modules in dozens of packages, I think.

I guess that's meant to encourage you to keep your modules small enough that you know what you're doing. At the smallest level, it would be a matter of: If in C++ you would declare Qwert to be a friend of Yuiop, then in D you put Qwert and Yuiop in the same module. You could implement the converse as well, but for a bunch of small classes it usually isn't worth it.
 It also adds to the problem that most people familiar with OO paradigm
 consider classes to be such encapsulation units.  Surprizes are
 inevitable.

I once came up with the idea a 'veryprivate' protection attribute that would do this, but I can't seem to find the post now. Stewart.
Jul 28 2009
parent Sergey Gromov <snake.scaly gmail.com> writes:
Wed, 29 Jul 2009 00:42:31 +0100, Stewart Gordon wrote:

 Sergey Gromov wrote:
 Mon, 27 Jul 2009 07:59:40 -0500, Andrei Alexandrescu wrote:
 
 Is it appropriate to define multiple classes, structs, templates, etc 
 within a single module? What considerations should inform the decision 
 regarding the placement of module boundaries?

bundle. The rule of thumb is, module provides the functionality, and it's up to the designer to decide what that entails.

That's the problem. On one hand, it's desirable to see a module as a functionality bundle. On the other hand, module is the smallest available unit of encapsulation. That is, if you have a class and change its private implementation, this may affect *anything* in the same module. Hence Tango's hundreds of modules in dozens of packages, I think.

I guess that's meant to encourage you to keep your modules small enough that you know what you're doing.

Yes you can do a finer-grained encapsulation, but then you end up without "functionality bundle." And even if you create a collective import module I think there is no way to make it stand out as such. Though this is no different from C.
 At the smallest level, it would be a matter of: If in C++ you would 
 declare Qwert to be a friend of Yuiop, then in D you put Qwert and Yuiop 
 in the same module.  You could implement the converse as well, but for a 
 bunch of small classes it usually isn't worth it.

This is understandable.
 It also adds to the problem that most people familiar with OO paradigm
 consider classes to be such encapsulation units.  Surprizes are
 inevitable.

I once came up with the idea a 'veryprivate' protection attribute that would do this, but I can't seem to find the post now.

I wonder how much code will break if "private" keyword becomes really private. You can always implement tight coupling with package visibility which C++ lacks.
Jul 28 2009
prev sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Mon, Jul 27, 2009 at 10:58 AM, Andrei
Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:
 Bill Baxter wrote:
 The requirement to add the post-blit is a little annoying. =A0If a
 ValueContainer is made fundamental and it is built initially out of
 components with value semantics, then no explicit post-blit will be
 necessary. =A0However if you go the other way and start with a
 RefContainer then a post-blit will be necessary when you implement the
 ValueContainer based on it. =A0 So to me it seems a little better to go
 with the ValueContainer as the base implementation.

I think it's a matter of the most frequent use. I used to be a staunch supporter of values for containers, until Walter revealed to me that in S=

 you have value containers to constantly worry about adding the "&"
 everywhere. Implicitly copying containers is 95% of cases an error in C++
 code. Witness all that advice about pass by reference etc. So why make th=

 the default, one might ask? Better go with the flow and make the default
 case the easy case (reference), and if someone does want a value, have th=

 write Value!Container.

Hmm. Yeh, when passing containers around, 95% or more probably should be by ref. However when creating composite objects I'd say the percentage of time you want by ref is much lower. But I guess even then, by-ref composition isn't necessarily bad (as most VM languages show). --bb
Jul 27 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 22 Jul 2009 21:55:54 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 I think we'd need at a minimum:

what are your opinions on the I/O subsystem? I think a lot of performance/features could be gained by using D-based buffered I/O instead of the C standard lib. Tango is pretty much a testament to that... -Steve
Jul 22 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 22 Jul 2009 22:03:56 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Steven Schveighoffer wrote:
 On Wed, 22 Jul 2009 21:55:54 -0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 I think we'd need at a minimum:

performance/features could be gained by using D-based buffered I/O instead of the C standard lib. Tango is pretty much a testament to that...

Better speed is always nice, so it would be great to see some work in that direction. What are the specific shortcomings that make using stdio unrecommended?

For one, you are not limited to the standard C libraries buffering features, there are a lot of cool things you can do with buffers in D that just weren't imagined or possible back when the API for C's stdio was developed. Take a look at the docs for Tango's buffering system (http://www.dsource.org/projects/tango/docs/stable/ look at tango.io.stream.Buffered and tango.io.device.Array). It's also one less abstraction layer to go through. -Steve
Jul 22 2009
prev sibling next sibling parent Jesse Phillips <jessekphillips gmail.com> writes:
On Wed, 22 Jul 2009 20:55:54 -0500, Andrei Alexandrescu wrote:

 I think we'd need at a minimum:
 
 * better networking library
 
 * containers
 
 * just a little linear algebra basics (i.e. well-defined storage that
 would allow us to interface with high-performance libraries)
 
 * at least some essentials for compile-time introspection (Shin, I'm
 still working on integrating WhiteHole and BlackHole; I want to expand
 your support function into a better fleshed and more general
 functionality)
 
 * a few functions here and there, e.g. readf
 
 * a complete rewrite of std.xml which is currently so slow it's not even
 funny
 
 * of course last but not least concurrency support
 
 * some more generic types a la std.typecons
 
 * flesh out std.complex which is to supplant the dying complex built-in
 type
 
 
 Andrei

These points were added to: http://www.prowiki.org/wiki4d/wiki.cgi?PhobosToDo Where they shall sit a long horrible life :P
Jul 22 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 27 Jul 2009 08:59:40 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Benji Smith wrote:
 For example, should the author of a container library prefer classes or  
 structs?

I've been struggling with this forever. I don't know. I don't even know whether reference or value semantics are best for containers. I don't know whether abstract container interfaces and container-independent code are a net win; experience with STL seems to say "don't" and experience with Java seems to say "ho-hum".

If you use classes with interfaces, then ranges can't be part of that equation. Since ranges are for the most part structs, and structs can't currently support interfaces, you can't make an interface that accepts a range. That's not to say ranges can't be part of classes, but they can't be part of interfaces. When making dcollections, I tried to separate out what should be part of the interface, and what should be specific to each class. What I found was that all the cursors (what you would call unsafe C++ iterators ;) ) could not be interface types, because they were structs for performance. However, I wanted to have the ability to have two different containers ineroperate seamlessly with eachother, as long as the types were the same. That suggested interfaces. What I ended up with is two ways to deal with container elements, opApply and cursors. I am biased of course, but I think it's the perfect combination of power, ease of use, and low cost of implementation. I still have yet to incorporate a range concept into my design, but I think it can be bolted on using a container with two cursors.
 Should other (non-container) modules accept container classes as  
 arguments? Or only container interfaces (if there are any such things)  
 or just ranges?

I think ranges should be preferred wherever applicable.

Ranges imply templated functions, which mean: - no polymorphism (although this requirement is questionable, most users don't derive from containers). - no usage of interfaces - each call may compile a new function (i.e. code bloat) There is something to be said with the power of opApply here, since you can do it with a class or interface, with very decent performance (only one virtual call). As I said earlier, a combination of both ranges and interfaces would be ideal. -Steve
Jul 27 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Mon, Jul 27, 2009 at 7:48 AM, Andrei
Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:
 Leandro Lucarella wrote:
 Andrei Alexandrescu, el 27 de julio a las 07:59 me escribiste:
 For example, should the author of a container library prefer classes or
 structs?

I've been struggling with this forever. I don't know. I don't even know whether reference or value semantics are best for containers. I don't know whether abstract container interfaces and container-independent code are a net win; experience with STL seems to say "don't" and experience with Java seems to say "ho-hum".

About values vs. reference semantics, I think reference is the best. Containers usually are big, and you don't want to copy them arround. I find myself using lots of references to pass containers arround in C++ and almost never use the copy() method of Python containers, so based on *my* experience, I'd say that reference as the default is the best approach.

Sounds convincing.

That's exactly right except for the times when it isn't and what you want is value semantics. I've just finished recently refactoring some C++ code that wasn't designed with copying in mind. Changing all the "float* data; int data_length;" members into std::vectors did the trick. The data they contained wasn't particularly large, and there was no real need to introduce the complexity that sharing it between copied instances would have created. --bb
Jul 27 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Mon, Jul 27, 2009 at 9:54 AM, Bill Baxter<wbaxter gmail.com> wrote:
 On Mon, Jul 27, 2009 at 7:48 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:
 Leandro Lucarella wrote:
 Andrei Alexandrescu, el 27 de julio a las 07:59 me escribiste:
 For example, should the author of a container library prefer classes =





 structs?

I've been struggling with this forever. I don't know. I don't even kno=




 whether reference or value semantics are best for containers. I don't
 know whether abstract container interfaces and container-independent
 code are a net win; experience with STL seems to say "don't" and
 experience with Java seems to say "ho-hum".

About values vs. reference semantics, I think reference is the best. Containers usually are big, and you don't want to copy them arround. I find myself using lots of references to pass containers arround in C+=



 and almost never use the copy() method of Python containers, so based o=



 *my* experience, I'd say that reference as the default is the best
 approach.

Sounds convincing.

That's exactly right except for the times when it isn't and what you want is value semantics. I've just finished recently refactoring some C++ code that wasn't designed with copying in mind. =A0Changing all the "float* data; int data_length;" members into std::vectors did the trick. =A0The data they contained wasn't particularly large, and there was no real need to introduce the complexity that sharing it between copied instances would have created.

So if I didn't make it clear -- my opinion is that whether containers should be by value or by ref depends on the circumstances and that choice should be left to the programmer. Thus the question becomes what's the easiest way to provide both? Three options I see: 1) Write ref classes and wrap them in structs 2) Write value structs and wrap them in classes 3) Write template mixins and mix them into both struct and class shells. I think given the existence of "alias this" for structs, 1 is actually the best-supported paradigm. I think with that you should be able to value-ize a by-ref container class just by writing a post-blitter to copy the contents. --bb
Jul 27 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Mon, Jul 27, 2009 at 10:01 AM, Bill Baxter<wbaxter gmail.com> wrote:
 On Mon, Jul 27, 2009 at 9:54 AM, Bill Baxter<wbaxter gmail.com> wrote:
 On Mon, Jul 27, 2009 at 7:48 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:
 Leandro Lucarella wrote:
 Andrei Alexandrescu, el 27 de julio a las 07:59 me escribiste:
 For example, should the author of a container library prefer classes=






 structs?

I've been struggling with this forever. I don't know. I don't even kn=





 whether reference or value semantics are best for containers. I don't
 know whether abstract container interfaces and container-independent
 code are a net win; experience with STL seems to say "don't" and
 experience with Java seems to say "ho-hum".

About values vs. reference semantics, I think reference is the best. Containers usually are big, and you don't want to copy them arround. I find myself using lots of references to pass containers arround in C=




 and almost never use the copy() method of Python containers, so based =




 *my* experience, I'd say that reference as the default is the best
 approach.

Sounds convincing.

That's exactly right except for the times when it isn't and what you want is value semantics. I've just finished recently refactoring some C++ code that wasn't designed with copying in mind. =A0Changing all the "float* data; int data_length;" members into std::vectors did the trick. =A0The data they contained wasn't particularly large, and there was no real need to introduce the complexity that sharing it between copied instances would have created.

So if I didn't make it clear -- my opinion is that whether containers should be by value or by ref depends on the circumstances and that choice should be left to the programmer. Thus the question becomes what's the easiest way to provide both? Three options I see: 1) Write ref classes and wrap them in structs 2) Write value structs and wrap them in classes 3) Write template mixins and mix them into both struct and class shells. I think given the existence of "alias this" for structs, 1 is actually the best-supported paradigm. I think with that you should be able to value-ize a by-ref container class just by writing a post-blitter to copy the contents.

Doh! I didn't realize classes could "alias this" too! So that would put 1 and 2 on equal footing in that regard. But maybe one argument in favor of structs is that it is easier to take value semantics and turn it into ref semantics. You just need to take a pointer. So this: class RefContainer { alias this valContainer_; this() { valContainer =3D new ValueContainer; } private: ValueContainer *valContainer; } gets you a basic working ref container out of a value container. To this you will probably want to add copy or clone type functionality. In constrast: struct ValueContainer { alias this refContainer_; this() { refContainer =3D new RefContainer; } // must have add post-blit to get value semantics private: RefContainer refContainer; } The requirement to add the post-blit is a little annoying. If a ValueContainer is made fundamental and it is built initially out of components with value semantics, then no explicit post-blit will be necessary. However if you go the other way and start with a RefContainer then a post-blit will be necessary when you implement the ValueContainer based on it. So to me it seems a little better to go with the ValueContainer as the base implementation. --bb
Jul 27 2009
prev sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Jul 28, 2009 at 4:42 PM, Stewart Gordon<smjg_1998 yahoo.com> wrote:
 Sergey Gromov wrote:
 Mon, 27 Jul 2009 07:59:40 -0500, Andrei Alexandrescu wrote:

 Is it appropriate to define multiple classes, structs, templates, etc
 within a single module? What considerations should inform the decision
 regarding the placement of module boundaries?

I think it's appropriate because many pieces of functionality come as a bundle. The rule of thumb is, module provides the functionality, and it=



 to the designer to decide what that entails.

That's the problem. =A0On one hand, it's desirable to see a module as a functionality bundle. =A0On the other hand, module is the smallest available unit of encapsulation. =A0That is, if you have a class and change its private implementation, this may affect *anything* in the same module. =A0Hence Tango's hundreds of modules in dozens of packages,=


 think.

I guess that's meant to encourage you to keep your modules small enough t=

 you know what you're doing.

 At the smallest level, it would be a matter of: If in C++ you would decla=

 Qwert to be a friend of Yuiop, then in D you put Qwert and Yuiop in the s=

 module. =A0You could implement the converse as well, but for a bunch of s=

 classes it usually isn't worth it.

 It also adds to the problem that most people familiar with OO paradigm
 consider classes to be such encapsulation units. =A0Surprizes are
 inevitable.

I once came up with the idea a 'veryprivate' protection attribute that wo=

 do this, but I can't seem to find the post now.

"super private" :-) Sorry I just can't let that old idea to call immutable "super const" die. It was just too darn cute. --bb
Jul 28 2009