www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Expanding Phobos from a flat hierachy

reply "Don" <don nospam.com> writes:
In the "Implementing Half Floats in D" thread, we seemed to have 
reached a consensus on two important points:
(a) Phobos should have a broad scope (rather than being small 
like the C standard library).
(b) The current flat structure of Phobos (every module in the 
root) does not scale to hundreds of modules.

It's not quite unanimous on (a), but seems to be close enough.

Together, (a) and (b) mean we need a plan. The sooner we can do 
it, the less painful it will be. But, on the other hand, it's 
something that we really don't want to get wrong.

Personal taste plays a huge role in this (Practically any 
structure can work, but we're looking for an arrangement which is 
intuitive and aesthetically pleasing to as many people as 
possible). It'll be most productive to stick to uncontroversial 
facts as long as possible.

It would be great if people with extensive experience in other 
languages could give a brief description of the structure of the 
corresponding libraries. (Also include pseudo-standard libraries, 
for example boost in C++).

1. How many total modules are there? How old is the library? What 
is the recent growth rate of the library (Percentage increase in 
size per year, over whatever time interval is reasonable for that 
library)?
2. How deep is the hierarchy? How many top-level branches are 
there?
3. How has the library dealt with obsolete functions and modules? 
Eg, are there entire top-level branches which are obsolete? How 
many stupid names are there (eg, "std.algorithm2") which were 
forced by the original design becoming obsolete?
4. How much of the library is successful/convincing? To make it a 
bit less subjective: how much of the library is commonly ignored, 
in favour of using third-party libraries instead? Which areas are 
failures in this way (eg, GUI, database, ...)? Which areas are 
the most often praised?

Maybe such a survey already exists, but I haven't been able to 
find an existing one.
Feb 05 2013
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, February 06, 2013 08:56:26 Don wrote:
 In the "Implementing Half Floats in D" thread, we seemed to have
 reached a consensus on two important points:
 (a) Phobos should have a broad scope (rather than being small
 like the C standard library).
 (b) The current flat structure of Phobos (every module in the
 root) does not scale to hundreds of modules.
 
 It's not quite unanimous on (a), but seems to be close enough.
 
 Together, (a) and (b) mean we need a plan. The sooner we can do
 it, the less painful it will be. But, on the other hand, it's
 something that we really don't want to get wrong.
 
 Personal taste plays a huge role in this (Practically any
 structure can work, but we're looking for an arrangement which is
 intuitive and aesthetically pleasing to as many people as
 possible). It'll be most productive to stick to uncontroversial
 facts as long as possible.

I agree that we should look at minimizing how many top-level modules we add in the future, but I don't think that it's generally worth trying to rearrange the modules that we already have. So, adding std.halffloat would be bad, but I'm not sure that it's worth doing something like moving std.uri into std.net. Even if we did, we'd be forced to leave std.uri around for quite some time (if not permanently, given Walter's attitude about such things), which negates the gain to some exent. You can also argue endlessly about what should be where in the hierarchy, which is needless bikeshedding. So, if we do any moving, we need to be sure that that's what we want to do and be done with it, and we shouldn't be moving new modules around. They need to go into the right spot when they're added, not be rearranged later.
 It would be great if people with extensive experience in other
 languages could give a brief description of the structure of the
 corresponding libraries. (Also include pseudo-standard libraries,
 for example boost in C++).

Java and C# would probably be good places to look, but I haven't dealt with either of them recently. It's easy enough to look at the docs though. They're well-organized. However, it should be kept in mind that both of those languages have the downside of being forced to have one public class per file (at least Java does - I don't remember for sure if that's true for C# or not), which can be good but is often overkill, which means that we wouldn't necessarily want to lay stuff out in quite the same way as they do, even if we agreed that they do a good job of it. - Jonathan M Davis
Feb 06 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/6/2013 12:15 AM, Jonathan M Davis wrote:
 I don't think that it's generally worth trying to rearrange
 the modules that we already have. So, adding std.halffloat would be bad, but
I'm
 not sure that it's worth doing something like moving std.uri into std.net.
 Even if we did, we'd be forced to leave std.uri around for quite some time

At least the contents of std.uri can be nothing more than: module std.uri; public import std.net.uri;
Feb 06 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 08:56, Don wrote:
 In the "Implementing Half Floats in D" thread, we seemed to have reached
 a consensus on two important points:
 (a) Phobos should have a broad scope (rather than being small like the C
 standard library).
 (b) The current flat structure of Phobos (every module in the root) does
 not scale to hundreds of modules.

 It's not quite unanimous on (a), but seems to be close enough.

 Together, (a) and (b) mean we need a plan. The sooner we can do it, the
 less painful it will be. But, on the other hand, it's something that we
 really don't want to get wrong.

+1 This is long overdue. -- /Jacob Carlborg
Feb 06 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 08:56, Don wrote:

 It would be great if people with extensive experience in other languages
 could give a brief description of the structure of the corresponding
 libraries. (Also include pseudo-standard libraries, for example boost in
 C++).

 1. How many total modules are there? How old is the library? What is the
 recent growth rate of the library (Percentage increase in size per year,
 over whatever time interval is reasonable for that library)?

Tango has 365 modules according to the documentation for D1 (including the object module).
 2. How deep is the hierarchy? How many top-level branches are there?

These are spread over 9 packages in the top level "tango" package. It seems to have at most four levels of packages, including the top level "tango" package.
 3. How has the library dealt with obsolete functions and modules? Eg,
 are there entire top-level branches which are obsolete? How many stupid
 names are there (eg, "std.algorithm2") which were forced by the original
 design becoming obsolete?

There are no obsolete top level packages. As far as I know there are no stupid names. At one time they did remove the "collection" package and replaced it with a "container" package. It's not really a stupid name. There can be other cases like this where the optimal name is not used but there are no names like "std.algorithm2". -- /Jacob Carlborg
Feb 06 2013
prev sibling next sibling parent "Don" <don nospam.com> writes:
On Wednesday, 6 February 2013 at 08:16:38 UTC, Jonathan M Davis 
wrote:
 On Wednesday, February 06, 2013 08:56:26 Don wrote:
 In the "Implementing Half Floats in D" thread, we seemed to 
 have
 reached a consensus on two important points:
 (a) Phobos should have a broad scope (rather than being small
 like the C standard library).
 (b) The current flat structure of Phobos (every module in the
 root) does not scale to hundreds of modules.
 
 It's not quite unanimous on (a), but seems to be close enough.
 
 Together, (a) and (b) mean we need a plan. The sooner we can do
 it, the less painful it will be. But, on the other hand, it's
 something that we really don't want to get wrong.
 
 Personal taste plays a huge role in this (Practically any
 structure can work, but we're looking for an arrangement which 
 is
 intuitive and aesthetically pleasing to as many people as
 possible). It'll be most productive to stick to uncontroversial
 facts as long as possible.


 You can also argue endlessly about what should be where in
 the hierarchy, which is needless bikeshedding.

Yes, exactly. That's why I want to defer passing judgement on anything until we have a reasonable set of data. Just because a design works well for a particular language doesn't mean it would also be good for D, but it's still good to look at. I think it's more constructive to say "that design was tried in Ruby, but they abandoned it" than to say "I personally don't like it". OTOH, if three different languages have totally different structures, and all have been widely praised, then it's clear it's a personal preference issue. And if we can say, "all nine libraries we've looked at did it this way", then the argument to do the same thing is very strong.
Feb 06 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 6 February 2013 at 10:15:28 UTC, Don wrote:
 On Wednesday, 6 February 2013 at 08:16:38 UTC, Jonathan M Davis 
 wrote:
 On Wednesday, February 06, 2013 08:56:26 Don wrote:
 In the "Implementing Half Floats in D" thread, we seemed to 
 have
 reached a consensus on two important points:
 (a) Phobos should have a broad scope (rather than being small
 like the C standard library).
 (b) The current flat structure of Phobos (every module in the
 root) does not scale to hundreds of modules.
 
 It's not quite unanimous on (a), but seems to be close enough.
 
 Together, (a) and (b) mean we need a plan. The sooner we can 
 do
 it, the less painful it will be. But, on the other hand, it's
 something that we really don't want to get wrong.
 
 Personal taste plays a huge role in this (Practically any
 structure can work, but we're looking for an arrangement 
 which is
 intuitive and aesthetically pleasing to as many people as
 possible). It'll be most productive to stick to 
 uncontroversial
 facts as long as possible.


 You can also argue endlessly about what should be where in
 the hierarchy, which is needless bikeshedding.

Yes, exactly. That's why I want to defer passing judgement on anything until we have a reasonable set of data. Just because a design works well for a particular language doesn't mean it would also be good for D, but it's still good to look at. I think it's more constructive to say "that design was tried in Ruby, but they abandoned it" than to say "I personally don't like it". OTOH, if three different languages have totally different structures, and all have been widely praised, then it's clear it's a personal preference issue. And if we can say, "all nine libraries we've looked at did it this way", then the argument to do the same thing is very strong.

I don't know if this is related or not (I'd think it is), but if we were to re-think phobos' module organization, we may also want to think about how we want to deal with changing entire modules at once. I'm mostly thinking about things like "random2.d", "xmk2.d" or whatnot. Or it could be un-related. Just saying we might want to take this into account.
Feb 06 2013
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, February 06, 2013 11:28:25 monarch_dodra wrote:
 I don't know if this is related or not (I'd think it is), but if
 we were to re-think phobos' module organization, we may also want
 to think about how we want to deal with changing entire modules
 at once.
 
 I'm mostly thinking about things like "random2.d", "xmk2.d" or
 whatnot.

I believe that that's what we decided on previously. It just hasn't actually been done yet, because no one has implemented such a module since the decision was made. - Jonathan M Davis
Feb 06 2013
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/6/2013 6:58 PM, deadalnix wrote:
 Well Walter seems very worried about breaking code, but seems unable to adopt
 any practice the reduce its effect, and in the facts, my cod break at any new
 dmd release (and right now compile with none released one).

Here's the current list of regressions: http://d.puremagic.com/issues/buglist.cgi?query_format=advanced&bug_severity=regression&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED If what's causing your code to break isn't there, it is unlikely to get fixed.
Feb 07 2013
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02/07/2013 03:58 AM, deadalnix wrote:
 ... my cod break at any new dmd release (and right now compile with none
released one).

+1. (I keep reporting regressions though, dustmite works quite well.)
Feb 07 2013
prev sibling next sibling parent "rumbu" <rumbu rumbu.ro> writes:
On Wednesday, 6 February 2013 at 07:56:26 UTC, Don wrote:
 It would be great if people with extensive experience in other 
 languages could give a brief description of the structure of 
 the corresponding libraries. (Also include pseudo-standard 
 libraries, for example boost in C++).

I know very well the BCL in C#, but I don't think is possible to map existing D modules to the same hierarchy, because D is lacking the ideea of namespace. More than that, the way things are organized in BCL doesn't match the D phobos rules, only the System namespace in BCL contains more than 300 different types with thousands of functions. I don't think anybody will like a system.d file containing that number of elements. The class hierarchy in main BCL assembly (mscorlib.dll) is organized as follows (important classes - almost everything in C# is a class) from D point of view: - system - basic types, general purpose functions; - system.collections - object ranges; - system.collections.generic - generic ranges; - system.diagnostics - debug helpers; - system.globalization - locale support; - system.io - streams, files, readers, writers - system.reflection - runtime traits; - system.resources - Windows resource management; - system.runtime.interopservices - WinAPI; - system.runtime.serialization - runtime serialization; - system.security - general purpose security functions; - system.security.cryptography - encryption algorithms; - system.text - text encoding/decoding support - system.threading - threading support - system.threading.tasks - task based threading, asynchrony Another important assembly (lib will be the immediate equivalent in D) is the system.dll assembly containing following important namespaces: - system - general purpose types; - system.codedom - compiler tools. somehow equivalent to ctfe in D; - system.collections.concurrent - shared collections and algorithms - system.componentmodel - particular cases of interfaces, events, classes - system.configuration - user settings, reading registry, ini files or various forms of persiostent data; - system.diagnostics - profiling, performance counters, unit testing; - system.compression - well known compression algorithms and associated streams; - system.io.ports - general purpose port interaction functions; - system.net - various communication protocols - system.security - various authentication systems - system.text.regularexpressions - regex; This is not a comprehensive list, there are more assemblies in BCL(eg: system.core.dll assembly contains also some important classes like linq extensions or system.xml.dll which contains all xml related stuff), and the namespaces above were shortened (eg: in BCL C# has system.net.mail for SMTP/IMAP or system.net.mime namespace for mime encoding). Regarding code organization, C# compiler is not dependent of the file system, there are no files or packages, you can code any number of classes in the same file or you can even code a class in multiple files (partial classes).
Feb 06 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 6 February 2013 at 10:51:19 UTC, Jonathan M Davis 
wrote:
 On Wednesday, February 06, 2013 11:28:25 monarch_dodra wrote:
 I don't know if this is related or not (I'd think it is), but 
 if
 we were to re-think phobos' module organization, we may also 
 want
 to think about how we want to deal with changing entire modules
 at once.
 
 I'm mostly thinking about things like "random2.d", "xmk2.d" or
 whatnot.

I believe that that's what we decided on previously. It just hasn't actually been done yet, because no one has implemented such a module since the decision was made.

Andrei made a proposal to allow transforming a module into a package. It seems to me like the way to go for this. This is even possible to manage several versions of the lib using version that way. And different module can even import their own. Sound like a better idea than messing up with names.
Feb 06 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, February 06, 2013 12:06:18 deadalnix wrote:
 Andrei made a proposal to allow transforming a module into a
 package. It seems to me like the way to go for this.

Yes, but that's a different issue. It's one thing to take std.algorithm or std.datetime and turn them into packages while still allowing you to import std.algorithm or std.datetime as before. It's quite another to completely replace a module with another module. To do that, you need a new module. Anything else will break code. The question then is what to name the new module.
 This is even possible to manage several versions of the lib using
 version that way. And different module can even import their own.
 Sound like a better idea than messing up with names.

You could replace the entire module in-place, but doing so will break code, and Walter in particular is very much against that in general. Whatever we did would require allowing people to transition from one to the other and we might even have to leave the old one around permanently (which I don't like, but I'm sure that Walter would favor). Simply changing it with a new version of Phobos wouldn't cut it. - Jonathan M Davis
Feb 06 2013
prev sibling next sibling parent "Dicebot" <m.strashun gmail.com> writes:
AFAIR there was proposal of a "future" meta package for 
introducing new modules with time to adapt, similar to how it is 
done in few other languages.
Feb 06 2013
prev sibling next sibling parent reply Robert <jfanatiker gmx.at> writes:
On Wed, 2013-02-06 at 08:56 +0100, Don wrote:
 Eg, are there entire top-level branches which are obsolete? How 
 many stupid names are there (eg, "std.algorithm2") which were 

Well in order to avoid such "stupid" names, I think it would be a good idea to keep the standard library small. Include only the things that are important to be "standard" in order to ensure interoperability between libraries. Instead we should offer a central place for D libraries, like CPAN for perl or http://pypi.python.org/pypi for python. The benefits: - The base installation stays small, which might be important for everything that is not a PC. - Versioning is easier, because every library can exist in multiple versions, and every project can choose to which version it wants to link. (For backwards compatiblity) -> No stupid names, if the libraries are small, simply deprecate the complete library. -> In fact such a central place is already on my todo list. I have the idea of creating an infrastructure, where you can easily write D scripts, which may import arbitrary libraries from the repository. A tool like rdmd would then notice that the libraries are not installed locally, so it does install them before running the script. (I think of integration with http://openbuildservice.org/ for Linux based systems and some custom installer for Windows.) -> This way we would have a really convenient way for scripting in D and it would be easy to toy around with an idea, before creating an actual project. -> So instead of creating a single monster standard library, just offer a central place for D libraries and automatic dependency management. I hope that I find the time soon to implement such a thing, but it will be a huge amount of work and maybe this is the right occasion to ask, whether anyone else thinks this would be a good idea. Just some of my recent thoughts. Best regards, Robert
Feb 06 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-06 13:02, Robert wrote:
 On Wed, 2013-02-06 at 08:56 +0100, Don wrote:
 Eg, are there entire top-level branches which are obsolete? How
 many stupid names are there (eg, "std.algorithm2") which were

Well in order to avoid such "stupid" names, I think it would be a good idea to keep the standard library small. Include only the things that are important to be "standard" in order to ensure interoperability between libraries. Instead we should offer a central place for D libraries, like CPAN for perl or http://pypi.python.org/pypi for python.

Here you go: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D I will replace the Ruby code with D. -- /Jacob Carlborg
Feb 06 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/6/13 8:42 AM, Jacob Carlborg wrote:
 On 2013-02-06 13:02, Robert wrote:
 On Wed, 2013-02-06 at 08:56 +0100, Don wrote:
 Eg, are there entire top-level branches which are obsolete? How
 many stupid names are there (eg, "std.algorithm2") which were

Well in order to avoid such "stupid" names, I think it would be a good idea to keep the standard library small. Include only the things that are important to be "standard" in order to ensure interoperability between libraries. Instead we should offer a central place for D libraries, like CPAN for perl or http://pypi.python.org/pypi for python.

Here you go: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D I will replace the Ruby code with D.

That's awesome. I skimmed the docs and the capabilities are pretty much what I'd expect. We should push for integrating Orbit in the standard distro. That entails a formal proposal, community review and voting etc. etc. One question - since you mention replacing Ruby with D, I'm thinking it may be a good opportunity to just use D syntax everywhere. The arguments have been hashed already - a D user is supposed to know of all syntaxes that of D itself etc. As an example I was looking at https://github.com/jacob-carlborg/orbit/wiki/orbfile which has: orb "dwt", "0.5.3" # specifies an exact version orb "sqlite" # uses the latest version orb "orange", "> 0.0.1" # uses any version greater than "0.0.1", basically any comparison operator is allowed here orb "derelict", "~> 0.3.4" # uses any version above "0.3.4" that is "0.x.y", i.e. won't use any "1.x.y" version Nice enough but there's no shame about orb("dwt", "0.5.3"); // specifies an exact version orb("sqlite"); // uses the latest version orb("orange", "> 0.0.1"); // uses any version greater than "0.0.1", basically any comparison operator is allowed here orb("derelict", "~> 0.3.4"); // uses any version above "0.3.4" that is "0.x.y", i.e. won't use any "1.x.y" version The alternative orbspec https://github.com/jacob-carlborg/orbit/wiki/orbspec%20specification also has its own syntax: # orange.orbspec # name "orange" # inferred from the name of the orbspec version "1.0.0" author "Jacob Carlborg" type :library files << ["a.d", "b.di"] # an array of source files This is in a way worse because there's a tidbit of syntax for each element. I'm thinking: spec = "orange"; version = "1.0.0"; author = "Jacob Carlborg"; type = "library"; files = ["a.d", "b.di"]; // an array of source files This code can be imported into an environment that has the appropriate definitions. Hash table literals may also be also very useful. Thoughts? Andrei
Feb 06 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-06 15:03, Andrei Alexandrescu wrote:

 That's awesome. I skimmed the docs and the capabilities are pretty much
 what I'd expect. We should push for integrating Orbit in the standard
 distro. That entails a formal proposal, community review and voting etc.
 etc.

This tool has a couple of dependencies on my own libraries (available of course) and Tango. I'm not feeling so enthusiastic to remove these dependencies. https://github.com/jacob-carlborg/dstack https://github.com/jacob-carlborg/mambo
 One question - since you mention replacing Ruby with D, I'm thinking it
 may be a good opportunity to just use D syntax everywhere. The arguments
 have been hashed already - a D user is supposed to know of all syntaxes
 that of D itself etc.

That was what I was referring to.
 Thoughts?

I was thinking having basically the same syntax. One syntax I'm not sure what to do about is the hash literal syntax Ruby uses. orb "dwt", git: "git://github.com/jacob-carlborg/dwt.git" https://github.com/jacob-carlborg/orbit/wiki/Integration The above could be translated to: orb("dwt", ["git" : "git://github.com/jacob-carlborg/dwt.git"); That doesn't look very nice. Also I might need to be able pass values of different types in the same hash. Sure, Variant can be used in that case but that would look even more uglier. -- /Jacob Carlborg
Feb 06 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/6/13 11:09 AM, Jacob Carlborg wrote:
 On 2013-02-06 15:03, Andrei Alexandrescu wrote:

 That's awesome. I skimmed the docs and the capabilities are pretty much
 what I'd expect. We should push for integrating Orbit in the standard
 distro. That entails a formal proposal, community review and voting etc.
 etc.

This tool has a couple of dependencies on my own libraries (available of course) and Tango. I'm not feeling so enthusiastic to remove these dependencies. https://github.com/jacob-carlborg/dstack https://github.com/jacob-carlborg/mambo

I understand. The way I see this is as an motivation and opportunity to add the necessary functionality to Phobos. I may be uninformed, but the way I see it basic package management doesn't have to be very demanding on library functionality.
 One question - since you mention replacing Ruby with D, I'm thinking it
 may be a good opportunity to just use D syntax everywhere. The arguments
 have been hashed already - a D user is supposed to know of all syntaxes
 that of D itself etc.

That was what I was referring to.
 Thoughts?

I was thinking having basically the same syntax. One syntax I'm not sure what to do about is the hash literal syntax Ruby uses. orb "dwt", git: "git://github.com/jacob-carlborg/dwt.git" https://github.com/jacob-carlborg/orbit/wiki/Integration The above could be translated to: orb("dwt", ["git" : "git://github.com/jacob-carlborg/dwt.git"); That doesn't look very nice. Also I might need to be able pass values of different types in the same hash. Sure, Variant can be used in that case but that would look even more uglier.

I'd say go with strings throughout. For the package manager "compile-time" is pretty much same as "run-time" so no need for early typechecking. Andrei
Feb 06 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 18:22, Andrei Alexandrescu wrote:

 I understand. The way I see this is as an motivation and opportunity to
 add the necessary functionality to Phobos. I may be uninformed, but the
 way I see it basic package management doesn't have to be very demanding
 on library functionality.

No it doesn't need to have a demanding library functionality. But I have creating these libraries because they contain common code I use in several of my tools and libraries. I think duplicating that code just to avoid having any library dependencies would be quite foolish. The reason why I use Tango is that it contains functionality that is either missing or not good enough in Phobos. Also, remember that I started this project several years ago. -- /Jacob Carlborg
Feb 07 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-06 18:35, Jens Mueller wrote:

 How about using YAML/JSON?
 name: dwt
 source: git://github.com/jacob-carlborg/dwt.git

The reason is that it will come a need for having conditions, variables and similar in the YAML/JSON file. So instead of creating a new format, or extending YAML/JSON with conditions and similar it's just easier to use a fully fledged language instead. DSSS is a perfect example of that it's a bad idea to invent a new format. DSSS build scripts are basically an extended variant INI files. It do support some kind of conditions: [main.d] buildflags += -L-L. version (Windows) { # do stuff } It just happens that if you but the first brace on its own line, or skips the space between the closing parenthesis and the opening brace it can't parse the file. That's that things I like to avoid by using a fully fledged language. That's way I originally chose Ruby, because it can look just like JSON: foo = { a: 3, b: 4 } With Ruby you can get a syntax that is declarative, just like JSON/YAML. -- /Jacob Carlborg
Feb 07 2013
parent FG <home fgda.pl> writes:
On 2013-02-07 09:19, Jacob Carlborg wrote:
 That's way I originally chose Ruby, because it can look just like JSON:

 foo = {
    a: 3,
    b: 4
 }

This looks like JSON but doesn't look like Ruby. :)
Feb 07 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/6/2013 6:03 AM, Andrei Alexandrescu wrote:
 This is in a way worse because there's a tidbit of syntax for each element. I'm
 thinking:

 spec = "orange";
 version = "1.0.0";
 author = "Jacob Carlborg";
 type = "library";
 files = ["a.d", "b.di"]; // an array of source files

That's close enough to JSON to suggest: why not use JSON syntax?
Feb 06 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 22:09, Walter Bright wrote:

 That's close enough to JSON to suggest: why not use JSON syntax?

See: http://forum.dlang.org/thread/ugmacrokqghrrwpfovam forum.dlang.org?page=4#post-kevo37:241gcn:241:40digitalmars.com -- /Jacob Carlborg
Feb 07 2013
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
06-Feb-2013 17:42, Jacob Carlborg пишет:
 On 2013-02-06 13:02, Robert wrote:
 On Wed, 2013-02-06 at 08:56 +0100, Don wrote:
 Eg, are there entire top-level branches which are obsolete? How
 many stupid names are there (eg, "std.algorithm2") which were

Well in order to avoid such "stupid" names, I think it would be a good idea to keep the standard library small. Include only the things that are important to be "standard" in order to ensure interoperability between libraries. Instead we should offer a central place for D libraries, like CPAN for perl or http://pypi.python.org/pypi for python.

Here you go: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D

 I will replace the Ruby code with D.

Great! -- Dmitry Olshansky
Feb 06 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 18:15, Dmitry Olshansky wrote:

 I've been wondering what happened to it, looks fine I think.

I've been working on other projects for a while but now I've started to work on it again. -- /Jacob Carlborg
Feb 07 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/6/13 4:24 PM, David Nadlinger wrote:
 On Wednesday, 6 February 2013 at 18:56:49 UTC, Brad Anderson wrote:
 What is needed to get Orbit off the ground (Kepler's law, I guess,
 would be the joke answer)?

Somebody actually working on it. No offense, Jacob, but to me it seems like you have quite a few interesting projects (DVM, DStep, Orbit, …), and although you are usually quick to advertise them here, it seems that none of them is quite ready for prime time. Maybe concentrating your efforts on one of them would be a good idea? Otherwise, you'll inevitably leave behind a trail of broken expectations as you cannot realistically expect to support all of them at the same time… David

I think one demotivating factor for both Jacob and the loosely-defined core D team is that we quickly reach irreducible positions on fundamental matter. It took literally two years for Jacob to decide to do away with Ruby. (Two years for real: https://github.com/jacob-carlborg/orbit/issues/1) Now we have dependencies on https://github.com/jacob-carlborg/dstack and https://github.com/jacob-carlborg/mambo. I took a look. This codebase is entirely neat and I'm glad a combination of author's talent and D makes the code so easy on the eyes, but there's no way we can ever make this part of the official distribution. It's parallel to the canon and some of it dedicates hundreds of lines to duplicating Phobos functionality with minor distinctions in functionality. I'm looking at e.g. https://github.com/jacob-carlborg/mambo/blob/master/mambo/arguments/Arguments.d which I bet money does some cool things with command-line arguments, but I see no reason for using it over std.getopt. Similarly https://github.com/jacob-carlborg/mambo/blob/master/mambo/arguments/Formatter.d is yet another std.format. Then, https://github.com/jacob-carlborg/mambo/blob/master/mambo/core/Array.d seems to be in the midst of a retrofitting to use std.algorithm as much of what it does now is do algorithm-y work with slightly changed names. Again, this is all fine and I'm not criticizing. If the purpose is to build a package manager for one's own use and to amass a community of users around it, great. But if the purpose is to make Orbit _the_ official package manager of the D programming language, there's no equivalent but distinct libraries for command-line arguments, no beginsWith that does what startsWith does, etc. etc. etc. There are three ways I see out of this: (a) Jacob goes full bore and ports Orbit to Phobos, and proposes the generally useful parts of his library for inclusion in Phobos; (b) Jacob agrees for someone else to do a clean room implementation of his package manager design; (c) we agree with the status quo with the understanding there's no more dangling of Orbit as a possible official package manager. Andrei
Feb 06 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 23:00, Andrei Alexandrescu wrote:

I'll start with some top posting and summarize. I use what works. If 
that means using some third party library I have no problem with that. 
Why should I reimplement something and hope for inclusion into Phobos 
when there already exist a perfectly good and working implementation.

 I think one demotivating factor for both Jacob and the loosely-defined
 core D team is that we quickly reach irreducible positions on
 fundamental matter. It took literally two years for Jacob to decide to
 do away with Ruby. (Two years for real:
 https://github.com/jacob-carlborg/orbit/issues/1)

I don't understand the hate about Ruby. People have just suggested to use JSON in this thread. Ruby can use the same syntax: foo = { a: 3, b: 4 } Now we have
 dependencies on https://github.com/jacob-carlborg/dstack and
 https://github.com/jacob-carlborg/mambo. I took a look. This codebase is
 entirely neat and I'm glad a combination of author's talent and D makes
 the code so easy on the eyes, but there's no way we can ever make this
 part of the official distribution. It's parallel to the canon and some
 of it dedicates hundreds of lines to duplicating Phobos functionality
 with minor distinctions in functionality.

Some of these modules go back to 2007, back when I was using D1. Some are just different names for functions in Phobos which I understand what you're not thrilled about including. Some are/were missing functions that I included in my own library instead of contributing to Phobos. I don't have time to wait five months for hoping something is included into Phobos.
 I'm looking at e.g.
 https://github.com/jacob-carlborg/mambo/blob/master/mambo/arguments/Arguments.d
 which I bet money does some cool things with command-line arguments, but
 I see no reason for using it over std.getopt.

This is a wrapper around the arguments parser in Tango which and some syntax sugar. It also helps integrating Formatter, mentioned below. Similarly
 https://github.com/jacob-carlborg/mambo/blob/master/mambo/arguments/Formatter.d
 is yet another std.format. Then,
 https://github.com/jacob-carlborg/mambo/blob/master/mambo/core/Array.d
 seems to be in the midst of a retrofitting to use std.algorithm as much
 of what it does now is do algorithm-y work with slightly changed names.

Since you wrote that comment I assume you haven't even looked at it, just the name. This module is specifically designed to automatically generate a usage/help text from the Arguments module above. I can see no such functionality in std.format, am I wrong?
 Again, this is all fine and I'm not criticizing. If the purpose is to
 build a package manager for one's own use and to amass a community of
 users around it, great. But if the purpose is to make Orbit _the_
 official package manager of the D programming language, there's no
 equivalent but distinct libraries for command-line arguments, no
 beginsWith that does what startsWith does, etc. etc. etc.

Yes, this contains new names for functions already available in std.algorithm. It also contains some tweaks here and there, convenient functions and new functionality that is missing from Phobos.
 There are three ways I see out of this: (a) Jacob goes full bore and
 ports Orbit to Phobos, and proposes the generally useful parts of his
 library for inclusion in Phobos; (b) Jacob agrees for someone else to do
 a clean room implementation of his package manager design; (c) we agree
 with the status quo with the understanding there's no more dangling of
 Orbit as a possible official package manager.

Doing some renames and similar is one thing but I have no intention of reimplementing a lot of code that is working perfectly fine and hoping for a inclusion into Phobos that most likely won't happen. I've tried to get a review of my serialization library for several years, sure it was a form or pre-review with the promise of removing any Tango and D1 related code if it was accepted. Nobody was interested. Sure I got the usual comments, I got real feedback from a single person here, but no formal review. Thank god I didn't remove the Tango and D1 stuff, that would be been a lot of unnecessary work and wasted time. I tried to get a review several times, but I've lost interested now, since no one else was interested. I'm also using a high level testing framework called Cucumber (uses Ruby). I'm 99% sure that you have no interest in this. It's a perfect framework to do this kind of testing I'm using it for. It's already available, working good and are supported by editors. Running tests, syntax highlighting and so on. Why should I give up that? I'm also not a fan of putting unit tests inline. I prefer to have a completely separate directory structure for my tests. My unit tests also uses a simple unit testing library that allows to continue after a assertion failure and gives summary at the end if any tests failed. I'm pretty sure you don't want this either. It's already there, it's working fine. Why should I give up that? I've compiled a list below of things used in Orbit (directly or indirectly) that Phobos is lacking in. Please tell me if I'm wrong: * DStack - Is a library for helping structuring applications. It includes: * Handling of command line arguments * Components * Commands * Starting and initializing of the application * Configuration I plan to add more stuff here when needed * Serialization - Does not exist * ConfigMap - Does not exist. An associative array with opDispatch added to it. Allows to do things like: auto config = ConfigMap(); config.foo = "asd"; config.bar.foo.baz = "foobar"; assert(config.foo == "asd"); * XML - The XML module is slow and has a cumbersome API * Zip - I don't exactly remember what was wrong here. It either didn't work or I had to reimplement a lot of functionality already available in Tango. * Net - std.curl is probably a good module but it wasn't available when I started. I also adds another dependency. * std.process - I have not used it myself but I've heard it's not optimal * std.getopt - Doesn't support the following: * Required arguments * Restricting the values of a given argument * No way to automatically create a help/usage list out of the arguments * Handling positional arguments * No support for commands/action. That is "git commit", "commit" would be the command/action * Handle multiple command lines * Validation of the arguments Various convince functions: * any - Opposite of empty * last - Returns the last element of an array * map, find and any for associative arrays * Utility functions for handling UDA's * isBlank - Check if a value if blank. If it's a string it's blank if it's only contains whitespace. If it you can call .empty return that. If it's a value type it's not empty. If it's a reference type and it's null it's empty * isPresent - The opposite of isBlank * pluralize - Takes a string and a count. If the count is greater than 1 it converts the word in the string to plural CTFE: * format - A simple formatting function * indexOf * contains * hasField - Returns true if the given type has the given field * fieldsOf - Returns all the fields of the given type * TypeOfField - Returns the type of a field * nameOfFieldAt - Returns the name of the field at the given position * set/getValueOfField - Sets/gets the value of a field * newInstance - Returns a new instance based on the class name or ClassInfo. Can handle any class regardless of constructors * A couple of traits In the end it's all about what's working. -- /Jacob Carlborg
Feb 07 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 16:41, Robert wrote:

 :-) That is pretty much what I had in mind! Awesome! I would love to
 help with this. I would start with some writing down of my ideas and
 concepts I have in mind, if you are interested? If you are, where should
 we discuss such things? On this mailing list?

Yes, I would be interested. I don't know what would be the best communication channel.
 I will replace the Ruby code with D.

language with all/most of the benefits of interpreted languages, it would seem strange if the tool that made this possible was not written in D. It would suggest that this was not possible in the first place. So I think this is a good idea. Concise definitions could be made possible, either by importing stuff in an appropriate environment as Andrei suggested or if needed even via a DSL, with string mixin's. So I think D already offers all we need, for good concise configuration files?

As I said, I will replace Ruby with D, that includes the DSL. It's the only place where Ruby is used. The rest is D. -- /Jacob Carlborg
Feb 07 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 19:56, Brad Anderson wrote:

 I have a dream that someday phobos will just be a metapackage in a
 package manager that installs a set of core modules.

 What is needed to get Orbit off the ground (Kepler's law, I guess, would
 be the joke answer)?

As with most things, it's time. I have basically an hour per day to work on D. On weekends I can work more on D. Quite a lot of that time goes to reading these newsgroups. I'm also not religiously working every one of these hours or being 100% focused. -- /Jacob Carlborg
Feb 07 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-06 22:24, David Nadlinger wrote:

 Somebody actually working on it.

 No offense, Jacob, but to me it seems like you have quite a few
 interesting projects (DVM, DStep, Orbit, …), and although you are
 usually quick to advertise them here, it seems that none of them is
 quite ready for prime time.

That's very true, that I'm working on several projects. Saying that none of them are ready for prime time is not really fair. DVM - Has seen several releases and does what I intended it to do. DStep - Has seen one release. It could use more work but it basically does what it should. Orange - Has only seen one proper release. Could be faster and support more archives but this does what it's intended to do Orbit - This is not finished yet. But many things do work.
 Maybe concentrating your efforts on one of them would be a good idea?
 Otherwise, you'll inevitably leave behind a trail of broken expectations
 as you cannot realistically expect to support all of them at the same time…

The problem is that I need all these tools and libraries. If one wants something done it's best to do it yourself. Also it can be quite tiresome to work on the same project all the time. Especially if you get stuck, then it can be good to have something else to work on for a while. -- /Jacob Carlborg
Feb 07 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 02/06/2013 08:56 AM, Don wrote:
 In the "Implementing Half Floats in D" thread, we seemed to have reached a
 consensus on two important points:
 (a) Phobos should have a broad scope (rather than being small like the C
 standard library).
 (b) The current flat structure of Phobos (every module in the root) does not
 scale to hundreds of modules.

Possibly naive question -- assuming that one has a package of multiple modules, how does one go about importing the whole package? Seems relevant if Phobos is to become nested.
Feb 06 2013
prev sibling next sibling parent Robert <jfanatiker gmx.at> writes:
On Wed, 2013-02-06 at 14:42 +0100, Jacob Carlborg wrote:
 Here you go:
 
 https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D

:-) That is pretty much what I had in mind! Awesome! I would love to help with this. I would start with some writing down of my ideas and concepts I have in mind, if you are interested? If you are, where should we discuss such things? On this mailing list?
 
 I will replace the Ruby code with D.

language with all/most of the benefits of interpreted languages, it would seem strange if the tool that made this possible was not written in D. It would suggest that this was not possible in the first place. So I think this is a good idea. Concise definitions could be made possible, either by importing stuff in an appropriate environment as Andrei suggested or if needed even via a DSL, with string mixin's. So I think D already offers all we need, for good concise configuration files? Best regards, Robert
Feb 06 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Joseph Rushton Wakeling:

 Possibly naive question -- assuming that one has a package of 
 multiple modules, how does one go about importing the whole 
 package?  Seems relevant if Phobos is to become nested.

I think implementing this: http://d.puremagic.com/issues/show_bug.cgi?id=3603 Bye, bearophile
Feb 06 2013
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Jacob Carlborg wrote:
 On 2013-02-06 15:03, Andrei Alexandrescu wrote:
Thoughts?

I was thinking having basically the same syntax. One syntax I'm not sure what to do about is the hash literal syntax Ruby uses. orb "dwt", git: "git://github.com/jacob-carlborg/dwt.git" https://github.com/jacob-carlborg/orbit/wiki/Integration The above could be translated to: orb("dwt", ["git" : "git://github.com/jacob-carlborg/dwt.git"); That doesn't look very nice.

How about using YAML/JSON? name: dwt source: git://github.com/jacob-carlborg/dwt.git Jens
Feb 06 2013
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Wednesday, 6 February 2013 at 11:38:31 UTC, Dicebot wrote:
 AFAIR there was proposal of a "future" meta package for 
 introducing new modules with time to adapt, similar to how it 
 is done in few other languages.

I really like the idea of an experimental section of phobos. I've been thinking lately that the phobos review might be improved by doing something more like this: 1. Proposal 2. Review and vote of overall usefulness and whether it meets a certain quality requirement. 3. Module is placed in experimental section of phobos with strong warnings that it is a trial module and could be removed in a future update. 4. Following a set period of time after the module has been in a public release another vote is held for actual inclusion. 5. If yea, move to its final resting place, if nay remove it for the experimental section. People are much more likely to actually test something if it's right at their fingertips and they don't have to download it and set up module paths and all of that. This isn't particularly hard with D but I think the easier it is for people to do, the better the reviews and module quality will be. Also, giving a hard time period within which the module can be modified freely because nobody expects it to be set in stone would help a lot. BA
Feb 06 2013
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Wednesday, 6 February 2013 at 13:42:36 UTC, Jacob Carlborg 
wrote:
 On 2013-02-06 13:02, Robert wrote:
 On Wed, 2013-02-06 at 08:56 +0100, Don wrote:
 Eg, are there entire top-level branches which are obsolete? 
 How
 many stupid names are there (eg, "std.algorithm2") which were

Well in order to avoid such "stupid" names, I think it would be a good idea to keep the standard library small. Include only the things that are important to be "standard" in order to ensure interoperability between libraries. Instead we should offer a central place for D libraries, like CPAN for perl or http://pypi.python.org/pypi for python.

Here you go: https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D I will replace the Ruby code with D.

I have a dream that someday phobos will just be a metapackage in a package manager that installs a set of core modules. What is needed to get Orbit off the ground (Kepler's law, I guess, would be the joke answer)? BA
Feb 06 2013
prev sibling next sibling parent Robert <jfanatiker gmx.at> writes:
+1 from me. As long as we don't have a central repository, where we
could have a separate phobos-staging package or something. At the moment
I don't think that additions to phobos were tested much in real world
before being adopted. Such an experimental section could really help and
we should try to get things in there quickly, to boost the whole process
a bit.

Best regards,

Robert

On Wed, 2013-02-06 at 19:47 +0100, Brad Anderson wrote:
 I really like the idea of an experimental section of phobos. I've 
 been thinking lately that the phobos review might be improved by 
 doing something more like this:
 
 1. Proposal
 2. Review and vote of overall usefulness and whether it meets a 
 certain quality requirement.
 3. Module is placed in experimental section of phobos with strong 
 warnings that it is a trial module and could be removed in a 
 future update.
 4. Following a set period of time after the module has been in a 
 public release another vote is held for actual inclusion.
 5. If yea, move to its final resting place, if nay remove it for 
 the experimental section.

Feb 06 2013
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Wednesday, 6 February 2013 at 21:10:06 UTC, Walter Bright 
wrote:
 On 2/6/2013 6:03 AM, Andrei Alexandrescu wrote:
 This is in a way worse because there's a tidbit of syntax for 
 each element. I'm
 thinking:

 spec = "orange";
 version = "1.0.0";
 author = "Jacob Carlborg";
 type = "library";
 files = ["a.d", "b.di"]; // an array of source files

That's close enough to JSON to suggest: why not use JSON syntax?

There is a competing project doing just that: https://github.com/rejectedsoftware/dub (doesn't look like it's been posted yet?)
Feb 06 2013
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Wednesday, 6 February 2013 at 18:56:49 UTC, Brad Anderson 
wrote:
 What is needed to get Orbit off the ground (Kepler's law, I 
 guess, would be the joke answer)?

Somebody actually working on it. No offense, Jacob, but to me it seems like you have quite a few interesting projects (DVM, DStep, Orbit, …), and although you are usually quick to advertise them here, it seems that none of them is quite ready for prime time. Maybe concentrating your efforts on one of them would be a good idea? Otherwise, you'll inevitably leave behind a trail of broken expectations as you cannot realistically expect to support all of them at the same time… David
Feb 06 2013
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Wednesday, 6 February 2013 at 21:16:01 UTC, Jakob Ovrum wrote:
 On Wednesday, 6 February 2013 at 21:10:06 UTC, Walter Bright 
 wrote:
 On 2/6/2013 6:03 AM, Andrei Alexandrescu wrote:
 This is in a way worse because there's a tidbit of syntax for 
 each element. I'm
 thinking:

 spec = "orange";
 version = "1.0.0";
 author = "Jacob Carlborg";
 type = "library";
 files = ["a.d", "b.di"]; // an array of source files

That's close enough to JSON to suggest: why not use JSON syntax?

There is a competing project doing just that: https://github.com/rejectedsoftware/dub (doesn't look like it's been posted yet?)

I asked Sönke about possibly proposing this as the official D package manager awhile back when Dub was called VPM. You can find his answer here: http://news.rejectedsoftware.com/groups/rejectedsoftware.vibed/thread/7/ BA
Feb 06 2013
prev sibling next sibling parent Robert <jfanatiker gmx.at> writes:
Ha! Great, I missed that. I thought it was vibe.d specific.

Thanks!
On Wed, 2013-02-06 at 22:15 +0100, Jakob Ovrum wrote:
 There is a competing project doing just that:
 
 https://github.com/rejectedsoftware/dub

Feb 06 2013
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Wednesday, 6 February 2013 at 21:55:53 UTC, Brad Anderson 
wrote:
 On Wednesday, 6 February 2013 at 21:16:01 UTC, Jakob Ovrum 
 wrote:
 On Wednesday, 6 February 2013 at 21:10:06 UTC, Walter Bright 
 wrote:
 On 2/6/2013 6:03 AM, Andrei Alexandrescu wrote:
 This is in a way worse because there's a tidbit of syntax 
 for each element. I'm
 thinking:

 spec = "orange";
 version = "1.0.0";
 author = "Jacob Carlborg";
 type = "library";
 files = ["a.d", "b.di"]; // an array of source files

That's close enough to JSON to suggest: why not use JSON syntax?

There is a competing project doing just that: https://github.com/rejectedsoftware/dub (doesn't look like it's been posted yet?)

I asked Sönke about possibly proposing this as the official D package manager awhile back when Dub was called VPM. You can find his answer here: http://news.rejectedsoftware.com/groups/rejectedsoftware.vibed/thread/7/ BA

Reading more, it looks like Dub is his attempt to make VPM less Vibe.d specific and more for general D use. It even generates VisualD project files. Cool stuff. BA
Feb 06 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, February 06, 2013 15:12:59 Joseph Rushton Wakeling wrote:
 On 02/06/2013 08:56 AM, Don wrote:
 In the "Implementing Half Floats in D" thread, we seemed to have reached a
 consensus on two important points:
 (a) Phobos should have a broad scope (rather than being small like the C
 standard library).
 (b) The current flat structure of Phobos (every module in the root) does
 not scale to hundreds of modules.

Possibly naive question -- assuming that one has a package of multiple modules, how does one go about importing the whole package? Seems relevant if Phobos is to become nested.

There are a couple of DIPS which relate to that (15 and 16 IIRC), but I would point out that in languages which _do_ permit that (e.g. Java), it's fairly common to consider it bad practice and insist on importing every module explicitly. - Jonathan M Davis
Feb 06 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 6 February 2013 at 11:33:20 UTC, Jonathan M Davis 
wrote:
 On Wednesday, February 06, 2013 12:06:18 deadalnix wrote:
 Andrei made a proposal to allow transforming a module into a
 package. It seems to me like the way to go for this.

Yes, but that's a different issue. It's one thing to take std.algorithm or std.datetime and turn them into packages while still allowing you to import std.algorithm or std.datetime as before. It's quite another to completely replace a module with another module. To do that, you need a new module. Anything else will break code. The question then is what to name the new module.

If we use the package trick as proposed, a possible solution is : std/datetime/v1.d : // Datetime version 1 std/datetime/v2.d : // Datetime version 2 std/datetime.d or std/datetime/package.d (or whatever) : version(DATETIME_V1) { public import std.datetime.v1; } else { public import std.datetime.v2; }
 You could replace the entire module in-place, but doing so will 
 break code,
 and Walter in particular is very much against that in general. 
 Whatever we did
 would require allowing people to transition from one to the 
 other and we might
 even have to leave the old one around permanently (which I 
 don't like, but I'm
 sure that Walter would favor). Simply changing it with a new 
 version of Phobos
 wouldn't cut it.

Well Walter seems very worried about breaking code, but seems unable to adopt any practice the reduce its effect, and in the facts, my cod break at any new dmd release (and right now compile with none released one).
Feb 06 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 6 February 2013 at 21:10:06 UTC, Walter Bright 
wrote:
 On 2/6/2013 6:03 AM, Andrei Alexandrescu wrote:
 This is in a way worse because there's a tidbit of syntax for 
 each element. I'm
 thinking:

 spec = "orange";
 version = "1.0.0";
 author = "Jacob Carlborg";
 type = "library";
 files = ["a.d", "b.di"]; // an array of source files

That's close enough to JSON to suggest: why not use JSON syntax?

Well if JSON syntax is better than D's why D don' use JSON syntax in the first place ?
Feb 06 2013
prev sibling next sibling parent reply Chad Joan <chadjoan gmail.com> writes:
On 02/06/2013 02:56 AM, Don wrote:
 In the "Implementing Half Floats in D" thread, we seemed to have reached
 a consensus on two important points:
 (a) Phobos should have a broad scope (rather than being small like the C
 standard library).
 (b) The current flat structure of Phobos (every module in the root) does
 not scale to hundreds of modules.

 It's not quite unanimous on (a), but seems to be close enough.

 Together, (a) and (b) mean we need a plan. The sooner we can do it, the
 less painful it will be. But, on the other hand, it's something that we
 really don't want to get wrong.

 ...

You ask for a lot of thorough research that I don't have time to provide at the moment. I just want to say "PLEASE YES". Phobos' module hierarchy is currently confusing at best, especially with various important elements being scattered across std.range, std.algorithm, std.array, and maybe another I'm forgetting. Although I can sort of see the logic behind this convention, it is very hard to think of a function that operates on ranges and automatically know which module to look in. I'm rambling, sorry. PLEASE YES!
Feb 06 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/6/2013 8:06 PM, Chad Joan wrote:
 Although I can sort of see the logic behind this convention, it is very hard to
 think of a function that operates on ranges and automatically know which module
 to look in.

Unfortunately, I don't think any reorg will help with that, as all functions that deal with input and output should use ranges.
Feb 07 2013
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 6 February 2013 at 21:07:24 UTC, Walter Bright 
wrote:
 On 2/6/2013 12:15 AM, Jonathan M Davis wrote:
 I don't think that it's generally worth trying to rearrange
 the modules that we already have. So, adding std.halffloat

 not sure that it's worth doing something like moving std.uri

 Even if we did, we'd be forced to leave std.uri around for

At least the contents of std.uri can be nothing more than: module std.uri; public import std.net.uri;

Just throwing in an idea, in case no one mentioned it yet: To avoid cluttering the Phobos directory structure, documentation, makefiles, etc. with redirect modules, the redirect modules can be placed in a completely separate directory (which would also have a "std" subdirectory), which would be added to sc.ini/dmd.conf's include path.
Feb 06 2013
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 06 Feb 2013 13:02:29 +0100
schrieb Robert <jfanatiker gmx.at>:

 On Wed, 2013-02-06 at 08:56 +0100, Don wrote:
 Eg, are there entire top-level branches which are obsolete? How 
 many stupid names are there (eg, "std.algorithm2") which were 

Well in order to avoid such "stupid" names, I think it would be a good idea to keep the standard library small. Include only the things that are important to be "standard" in order to ensure interoperability between libraries.

I think the opposite. When we think about more concepts in Phobos, like gui, serialization, cryptography and so on, we will be able to create the right amount of hierarchy with distinguishable names. I'm in favor of a big batteries included library.
 The benefits:
 - The base installation stays small, which might be important for
   everything that is not a PC.

That's why we have modules and minimize dependencies between them. Years ago when Java was introduced to mobile phones the big library was no problem, because non-essential parts could be removed. -- Marco
Feb 06 2013
prev sibling next sibling parent Robert <jfanatiker gmx.at> writes:
As Sönke Ludwig seems to work on something very similar, it might make
sense to join forces with him too. I will look into both implementations
next week, to see where we stand.

 
 As I said, I will replace Ruby with D, that includes the DSL. It's the 
 only place where Ruby is used. The rest is D.
 

Feb 07 2013
prev sibling next sibling parent Robert <jfanatiker gmx.at> writes:
On Thu, 2013-02-07 at 09:45 +0100, Jacob Carlborg wrote:
 Yes, I would be interested. I don't know what would be the best 
 communication channel.

Great! :-) I think for now, maybe this mailing list is not the worst place. (At least I don't know a better one) Best regards, Robert
Feb 07 2013
prev sibling next sibling parent Robert <jfanatiker gmx.at> writes:
 I think the opposite. When we think about more concepts in
 Phobos, like gui, serialization, cryptography and so on, we
 will be able to create the right amount of hierarchy with
 distinguishable names. I'm in favor of a big batteries
 included library.

Yeah, but I still think that breaking independent parts up in independent projects, will make the whole thing more manageable. If some of these projects happen to be within the std package and be installed by default, that is fine. Best regards, Robert
Feb 07 2013
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Wednesday, 6 February 2013 at 22:25:03 UTC, Brad Anderson 
wrote:
 Reading more, it looks like Dub is his attempt to make VPM less 
 Vibe.d specific and more for general D use.  It even generates 
 VisualD project files.

 Cool stuff.

 BA

Yeah, I'm leaning towards DUB for a few reasons. The implementation is written in idiomatic D2; it's much more likely to attract contributors this way. The online package registry is already up and is entirely powered by vibe.d and its ecosystem: we're eating our own dog food throughout. Also, using a data format (JSON, in this case) over a scripting language has its advantages. With DUB, there is only one piece of software you need to trust (which is DUB itself), as the package description file is not software. It's possible to do things like automatically building packages without having to vet a build script's quality. I'm not sure how sound this point really is, but it feels right (for what little it's worth...).
Feb 07 2013
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 07 Feb 2013 05:19:56 +0100
schrieb "Vladimir Panteleev" <vladimir thecybershadow.net>:

 On Wednesday, 6 February 2013 at 21:07:24 UTC, Walter Bright 
 wrote:
 On 2/6/2013 12:15 AM, Jonathan M Davis wrote:
 I don't think that it's generally worth trying to rearrange
 the modules that we already have. So, adding std.halffloat

 not sure that it's worth doing something like moving std.uri

 Even if we did, we'd be forced to leave std.uri around for

At least the contents of std.uri can be nothing more than: module std.uri; public import std.net.uri;

Just throwing in an idea, in case no one mentioned it yet: To avoid cluttering the Phobos directory structure, documentation, makefiles, etc. with redirect modules, the redirect modules can be placed in a completely separate directory (which would also have a "std" subdirectory), which would be added to sc.ini/dmd.conf's include path.

That's cool. Together with the other proposals I think we have enough infrastructure ready to solve all problems with module updates and moves. You can also remove that line from dmd.conf and add it only to project configurations with 'legacy' code. Since IDEs like Mono-D already scan available modules in the include paths for imports, this would reduce the noise a bit. -- Marco
Feb 07 2013
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 07 Feb 2013 04:02:29 +0100
schrieb "deadalnix" <deadalnix gmail.com>:

 Well if JSON syntax is better than D's why D don' use JSON syntax 
 in the first place ?

JSON is a well known format. The benefit is, parsers exist for every practical language and developers know by heart the syntax and can start editing. And mixing a data definition language with a programming language doesn't work here. Maybe if this was the LISP news group... :) -- Marco
Feb 07 2013
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 06 Feb 2013 23:06:06 -0500
schrieb Chad Joan <chadjoan gmail.com>:

 I just want to say "PLEASE YES".  Phobos' module hierarchy is currently 
 confusing at best, especially with various important elements being 
 scattered across std.range, std.algorithm, std.array, and maybe another 
 I'm forgetting.  Although I can sort of see the logic behind this 
 convention, it is very hard to think of a function that operates on 
 ranges and automatically know which module to look in.  I'm rambling, sorry.
 
 PLEASE YES!

It is true. I've had the same discussion on IRC a while ago. After letting off some steam, I had the idea to make a map of Phobos by functionality instead of module. Sometimes simple stuff confuses me: is .empty in std.array or std.range? And why is it not a standard property of arrays? The module borders seem washy in std.rangorithray. String formatting is another example with functions spread over conv/string/format for: 'xformat' (std.string), 'text' (std.conv) or 'formattedWrite' (std.format). Add manipulation to the list and the function you look for may even be generalized in std.algorithm, like 'startsWith'. String slicing/concatenation: 'stripRight' (std.string), 'startsWith' (std.algorithm), 'join' (std.array) Text I/O: 'readText' (std.file), 'byLine' (std.stdio) Temporary files: 'tmpfile' (std.stdio), 'tempDir' (std.file) Charsets: 'toMBSz' (std.windows.charset), 'transcode' (std.encoding), 'toUTF16' (std.utf) Memory streams: 'MemoryStream' (std.stream), 'OutBuffer' (std.outbuffer) ... and so on ... -- Marco
Feb 07 2013
prev sibling next sibling parent "Paul D. Anderson" <paul.d.removethis.anderson comcast.andthis.net> writes:
On Wednesday, 6 February 2013 at 07:56:26 UTC, Don wrote:
 In the "Implementing Half Floats in D" thread, we seemed to 
 have reached a consensus on two important points:
 (a) Phobos should have a broad scope (rather than being small 
 like the C standard library).
 (b) The current flat structure of Phobos (every module in the 
 root) does not scale to hundreds of modules.

 It's not quite unanimous on (a), but seems to be close enough.

 Together, (a) and (b) mean we need a plan. The sooner we can do 
 it, the less painful it will be. But, on the other hand, it's 
 something that we really don't want to get wrong.

 Personal taste plays a huge role in this (Practically any 
 structure can work, but we're looking for an arrangement which 
 is intuitive and aesthetically pleasing to as many people as 
 possible). It'll be most productive to stick to uncontroversial 
 facts as long as possible.

 It would be great if people with extensive experience in other 
 languages could give a brief description of the structure of 
 the corresponding libraries. (Also include pseudo-standard 
 libraries, for example boost in C++).

 1. How many total modules are there? How old is the library? 
 What is the recent growth rate of the library (Percentage 
 increase in size per year, over whatever time interval is 
 reasonable for that library)?
 2. How deep is the hierarchy? How many top-level branches are 
 there?
 3. How has the library dealt with obsolete functions and 
 modules? Eg, are there entire top-level branches which are 
 obsolete? How many stupid names are there (eg, 
 "std.algorithm2") which were forced by the original design 
 becoming obsolete?
 4. How much of the library is successful/convincing? To make it 
 a bit less subjective: how much of the library is commonly 
 ignored, in favour of using third-party libraries instead? 
 Which areas are failures in this way (eg, GUI, database, ...)? 
 Which areas are the most often praised?

 Maybe such a survey already exists, but I haven't been able to 
 find an existing one.

Java Standard Library: Total of 209 packages. 3 top-level packages: java ( 56 packages, 13 top level within java) javax (117 packages, 20 top level within javax) org ( 36 packages, 4 top-level within org) Max depth of package tree: 5 Duplicated Names: java.io and java.nio (new io) java.net and javax.net java.lang and javax.lang Note that javax was initially intended to be an experimental library where its packages would get moved to the java library eventually. But the first time they tried to move a package from javax to java there were a lot of complaints. Too much broken code. So the experimental library become an addition to the standard library. Paul
Feb 07 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, February 07, 2013 13:17:20 Jacob Carlborg wrote:
 On 2013-02-06 18:22, Andrei Alexandrescu wrote:
 I understand. The way I see this is as an motivation and opportunity to
 add the necessary functionality to Phobos. I may be uninformed, but the
 way I see it basic package management doesn't have to be very demanding
 on library functionality.

No it doesn't need to have a demanding library functionality. But I have creating these libraries because they contain common code I use in several of my tools and libraries. I think duplicating that code just to avoid having any library dependencies would be quite foolish. The reason why I use Tango is that it contains functionality that is either missing or not good enough in Phobos. Also, remember that I started this project several years ago.

All of which is fine as long as it's a 3rd party project. I think that Andrei's point is that if you have any interest in it being D's official package manager, it needs to be changed to use Phobos only, even if that requires adding or improving functionality in Phobos. An official package manager can't rely on 3rd party libraries or tools. So, either Orbit needs to be updated to not depend on anything beyond D's standard library, or it should stop being presented as a possible official package manager. That's not saying anything bad about the project or the work that you've done. It's just saying that it does not currently fit the requirements for an official project. - Jonathan M Davis
Feb 07 2013
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Wednesday, 6 February 2013 at 18:47:35 UTC, Brad Anderson 
wrote:
 On Wednesday, 6 February 2013 at 11:38:31 UTC, Dicebot wrote:
 AFAIR there was proposal of a "future" meta package for 
 introducing new modules with time to adapt, similar to how it 
 is done in few other languages.

I really like the idea of an experimental section of phobos. I've been thinking lately that the phobos review might be improved by doing something more like this: 1. Proposal 2. Review and vote of overall usefulness and whether it meets a certain quality requirement. 3. Module is placed in experimental section of phobos with strong warnings that it is a trial module and could be removed in a future update. 4. Following a set period of time after the module has been in a public release another vote is held for actual inclusion. 5. If yea, move to its final resting place, if nay remove it for the experimental section. People are much more likely to actually test something if it's right at their fingertips and they don't have to download it and set up module paths and all of that. This isn't particularly hard with D but I think the easier it is for people to do, the better the reviews and module quality will be. Also, giving a hard time period within which the module can be modified freely because nobody expects it to be set in stone would help a lot. BA

Just one last idea I had I'd like to share before I forget about it. I think module reviews should, in part, come in the form of pull requests. High level reviews need discussion so they would still be done here on the forums but simple documentation fixes and other tweaks would be better handled by pull requests on the module's home repo. Seems silly that people will type up all of their suggestions and then the module author will slowly go through the review responses looking for changes he should be making when actually making the changes and sending a pull request would take less time and as well as some of the huge burden off of the author of the module. BA
Feb 07 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/6/13, Don <don nospam.com> wrote:
 (b) The current flat structure of Phobos (every module in the
 root) does not scale to hundreds of modules.

I just want to add that we're still handicapped by D's access attributes preventing us from moving private functionality into a separate module.
Feb 07 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, February 07, 2013 23:56:30 Andrej Mitrovic wrote:
 On 2/6/13, Don <don nospam.com> wrote:
 (b) The current flat structure of Phobos (every module in the
 root) does not scale to hundreds of modules.

I just want to add that we're still handicapped by D's access attributes preventing us from moving private functionality into a separate module.

And why is this a handicap? They're private, not part of the API, so they should totally irrelevant to how modules are laid out. - Jonathan M Davis
Feb 07 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 7 February 2013 at 22:56:39 UTC, Andrej Mitrovic 
wrote:
 On 2/6/13, Don <don nospam.com> wrote:
 (b) The current flat structure of Phobos (every module in the
 root) does not scale to hundreds of modules.

I just want to add that we're still handicapped by D's access attributes preventing us from moving private functionality into a separate module.

That is the whole point of private. If something is imported from elsewhere and used, by definition, it isn't private.
Feb 07 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/8/13, deadalnix <deadalnix gmail.com> wrote:
 That is the whole point of private. If something is imported from
 elsewhere and used, by definition, it isn't private.

 And why is this a handicap? They're private, not part of the API, so they
 should totally irrelevant to how modules are laid out.

Have you ever tried to used a project layout where you implement private (e.g. OS-specific) functionality into their own subpackages, which should only be accessible to the library and not the user? Because this is impossible to do right now. IOW if you have: ./std/algorithm.d ./std/range.d Say that both modules require some private internal template called "MagicTemplate". Personally I find it ideal to use internal packages for this, e.g.: ./std/algorithm.d ./std/range.d ./std/internal/templates.d -- contains MagicTemplate, imported by algorithm and range But this forces you to make MagicTemplate public, as package access won't work in this case. Maybe "handicapped" was the wrong word for this. You could put the template in algorithm.d or range.d and have one module import the other (which has its own problems, e.g. with module constructors), or you could put the internal.d module in the std package and set package access to MagicTemplate: ./std/algorithm.d ./std/range.d ./std/internal.d -- contains MagicTemplate with package access But I prefer to have internal modules in subpackages, which is impossible to do right now in D due to how access specifiers work (i.e. everything ends up being public). Anyway it's not a blocker but it's worth thinking about.
Feb 08 2013
prev sibling next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Friday, 8 February 2013 at 13:14:09 UTC, Andrej Mitrovic wrote:
 ...

As far as I can see it, this will be automatically settled once package import DIP is done. Because then you can have hierarchy like this: std/algorithm/_.d std/algorithm/implementation.d ..and use "package" protection attribute.
Feb 08 2013
prev sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Friday, 8 February 2013 at 13:14:09 UTC, Andrej Mitrovic wrote:
 But I prefer to have internal modules in subpackages, which is
 impossible to do right now in D due to how access specifiers 
 work
 (i.e. everything ends up being public). Anyway it's not a 
 blocker but
 it's worth thinking about.

Considering imports can be private, I don't see it as a big problem. Surely the user won't have any expectations if he chooses to explicitly import a module that is clearly labeled internal.
Feb 08 2013